diff --git a/README_cn.md b/README_cn.md
index 1ecb023d1d6b..f6780f8d2e83 100644
--- a/README_cn.md
+++ b/README_cn.md
@@ -30,6 +30,9 @@
**PaddleNLP**是一款**简单易用**且**功能强大**的自然语言处理开发库。聚合业界**优质预训练模型**并提供**开箱即用**的开发体验,覆盖NLP多场景的模型库搭配**产业实践范例**可满足开发者**灵活定制**的需求。
## News 📢
+
+* 🔥 **2022.11.12 PaddleNLP新增AutoPrompt自动化提示功能,登顶FewCLUE小样本学习榜单!**
+ * 🥇 PaddleNLP 团队开源了 **AutoPrompt** 方案,基于开源的文心 ERNIE 预训练语言模型 ,结合了领域预训练和自动化提示学习技术,以291M 参数量的模型在小样本权威学习榜单 FewCLUE 排名第一,[详见](https://mp.weixin.qq.com/s/_JPiAzFA1f0BZ0igdv-EKA)。
* 🔥 **2022.10.27 发布 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2)**
* NLG能力扩充:新增📄[**基于Pegasus的中文文本摘要方案**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus),效果领先;新增❓[**问题生成解决方案**](./examples/question_generation),提供基于业界领先模型UNIMO-Text和大规模多领域问题生成数据集训练的通用问题生成预训练模型。均支持Taskflow一键调用,支持FasterGeneration高性能推理,训练推理部署全流程打通。
* 发布 🖼[**PPDiffusers**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers):支持跨模态(如图像与语音)训练和推理的扩散模型(Diffusion Model)工具箱,可快速体验、二次开发 **Stable Diffusion**,持续支持更多模型。
@@ -234,7 +237,7 @@ PaddleNLP针对信息抽取、语义检索、智能问答、情感分析等高
### 高性能分布式训练与推理
-#### ⚡ FasterTokenizer:高性能文本处理库
+#### ⚡ FastTokenizer:高性能文本处理库
![](https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png)
@@ -244,7 +247,7 @@ PaddleNLP针对信息抽取、语义检索、智能问答、情感分析等高
AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_faster=True)
```
-为了实现更极致的模型部署性能,安装FastTokenizers后只需在`AutoTokenizer` API上打开 `use_faster=True`选项,即可调用C++实现的高性能分词算子,轻松获得超Python百余倍的文本处理加速,更多使用说明可参考[FasterTokenizer文档](./faster_tokenizer)。
+为了实现更极致的模型部署性能,安装FastTokenizers后只需在`AutoTokenizer` API上打开 `use_faster=True`选项,即可调用C++实现的高性能分词算子,轻松获得超Python百余倍的文本处理加速,更多使用说明可参考[FastTokenizer文档](./fast_tokenizer)。
#### ⚡️ FasterGeneration:高性能生成加速库
diff --git a/README_en.md b/README_en.md
index ec76f859a9a5..3cc50086f8cd 100644
--- a/README_en.md
+++ b/README_en.md
@@ -30,6 +30,8 @@
## News 📢
+* 🔥 **2022.11.12 PaddleNLP added AutoPrompt and won the first place in FewCLUE!**
+ * 🥇 The PaddleNLP team has open-sourced the **AutoPrompt** solution, which is based on the open-source Wenxin ERNIE pre-training language model, combined with domain pre-training and automated prompt learning technology, and ranked first in FewCLUE (a authoritative few-sample learning contest) with a model with 291M parameters. [see details](https://mp.weixin.qq.com/s/_JPiAzFA1f0BZ0igdv-EKA).
* 🔥 **2022.10.27 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2) Released!**
* NLG Upgrade: 📄 Release [**Solution of Text Summarization**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus) based on Pegasus;❓ Release [**Solution of Problem Generation**](./examples/question_generation), providing **general problem generation pre-trained model** based on Baidu's UNIMO Text and large-scale multi domain problem generation dataset。Supporting high-performance inference ability based on FasterGeneration , and covering the whole process of training , inference and deployment.
* 🔥 **2022.10.14 [PaddleNLP v2.4.1](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.1) Released!**
diff --git a/applications/neural_search/recall/in_batch_negative/README.md b/applications/neural_search/recall/in_batch_negative/README.md
index ed04bd15b4e9..aa07d6908480 100644
--- a/applications/neural_search/recall/in_batch_negative/README.md
+++ b/applications/neural_search/recall/in_batch_negative/README.md
@@ -229,6 +229,9 @@ python -u -m paddle.distributed.launch --gpus "0,1,2,3" \
* `recall_num`: 对 1 个文本召回的相似文本数量
* `similar_text_pair_file`: 由相似文本对构成的评估集
* `corpus_file`: 召回库数据 corpus_file
+* `use_recompute`: 使用Recompute策略,用于节省显存,是一种以时间换空间的技术
+* `use_gradient_cache`: 使用Gradient Cache策略,用于节省显存,是一种以时间换空间的技术
+* `chunk_numbers`: 使用Gradient Cache策略的参数,表示的是同一个批次的样本分几次执行
也可以使用bash脚本:
diff --git a/applications/neural_search/recall/in_batch_negative/batch_negative/model.py b/applications/neural_search/recall/in_batch_negative/batch_negative/model.py
index 911fe0b4d360..050beb62f613 100644
--- a/applications/neural_search/recall/in_batch_negative/batch_negative/model.py
+++ b/applications/neural_search/recall/in_batch_negative/batch_negative/model.py
@@ -60,14 +60,14 @@ def forward(self,
title_cls_embedding,
transpose_y=True)
- # substract margin from all positive samples cosine_sim()
+ # Substract margin from all positive samples cosine_sim()
margin_diag = paddle.full(shape=[query_cls_embedding.shape[0]],
fill_value=self.margin,
dtype=paddle.get_default_dtype())
cosine_sim = cosine_sim - paddle.diag(margin_diag)
- # scale cosine to ease training converge
+ # Scale cosine to ease training converge
cosine_sim *= self.sacle
labels = paddle.arange(0, query_cls_embedding.shape[0], dtype='int64')
@@ -76,3 +76,56 @@ def forward(self,
loss = F.cross_entropy(input=cosine_sim, label=labels)
return loss
+
+
+class SemanticIndexCacheNeg(SemanticIndexBase):
+
+ def __init__(self,
+ pretrained_model,
+ dropout=None,
+ margin=0.3,
+ scale=30,
+ output_emb_size=None):
+ super().__init__(pretrained_model, dropout, output_emb_size)
+ self.margin = margin
+ # Used scaling cosine similarity to ease converge
+ self.sacle = scale
+
+ def forward(self,
+ query_input_ids,
+ title_input_ids,
+ query_token_type_ids=None,
+ query_position_ids=None,
+ query_attention_mask=None,
+ title_token_type_ids=None,
+ title_position_ids=None,
+ title_attention_mask=None):
+
+ query_cls_embedding = self.get_pooled_embedding(query_input_ids,
+ query_token_type_ids,
+ query_position_ids,
+ query_attention_mask)
+
+ title_cls_embedding = self.get_pooled_embedding(title_input_ids,
+ title_token_type_ids,
+ title_position_ids,
+ title_attention_mask)
+
+ cosine_sim = paddle.matmul(query_cls_embedding,
+ title_cls_embedding,
+ transpose_y=True)
+
+ # Substract margin from all positive samples cosine_sim()
+ margin_diag = paddle.full(shape=[query_cls_embedding.shape[0]],
+ fill_value=self.margin,
+ dtype=cosine_sim.dtype)
+
+ cosine_sim = cosine_sim - paddle.diag(margin_diag)
+
+ # Scale cosine to ease training converge
+ cosine_sim *= self.sacle
+
+ labels = paddle.arange(0, query_cls_embedding.shape[0], dtype='int64')
+ labels = paddle.reshape(labels, shape=[-1, 1])
+
+ return [cosine_sim, labels, query_cls_embedding, title_cls_embedding]
diff --git a/applications/neural_search/recall/in_batch_negative/scripts/run_build_index.sh b/applications/neural_search/recall/in_batch_negative/scripts/run_build_index.sh
index 857302c334a1..9920a045b9dc 100755
--- a/applications/neural_search/recall/in_batch_negative/scripts/run_build_index.sh
+++ b/applications/neural_search/recall/in_batch_negative/scripts/run_build_index.sh
@@ -1,6 +1,20 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
# GPU version
root_dir="checkpoints/inbatch"
-python -u -m paddle.distributed.launch --gpus "3" --log_dir "recall_log/" \
+python -u -m paddle.distributed.launch --gpus "0" --log_dir "recall_log/" \
recall.py \
--device gpu \
--recall_result_dir "recall_result_dir" \
@@ -11,7 +25,7 @@ python -u -m paddle.distributed.launch --gpus "3" --log_dir "recall_log/" \
--hnsw_ef 100 \
--batch_size 64 \
--output_emb_size 256\
- --max_seq_length 60 \
+ --max_seq_length 64 \
--recall_num 50 \
--similar_text_pair "recall/dev.csv" \
--corpus_file "recall/corpus.csv"
diff --git a/applications/neural_search/recall/in_batch_negative/train_batch_neg.py b/applications/neural_search/recall/in_batch_negative/train_batch_neg.py
index 10bead311455..fe48bd49ffbc 100644
--- a/applications/neural_search/recall/in_batch_negative/train_batch_neg.py
+++ b/applications/neural_search/recall/in_batch_negative/train_batch_neg.py
@@ -17,14 +17,17 @@
import time
import numpy as np
import paddle
+import paddle.nn.functional as F
from functools import partial
+
from paddlenlp.utils.log import logger
from paddlenlp.data import Tuple, Pad
from paddlenlp.datasets import load_dataset, MapDataset
from paddlenlp.transformers import AutoModel, AutoTokenizer
from paddlenlp.transformers import LinearDecayWithWarmup
+
from base_model import SemanticIndexBase
-from batch_negative.model import SemanticIndexBatchNeg
+from batch_negative.model import SemanticIndexBatchNeg, SemanticIndexCacheNeg
from data import read_text_pair, convert_example, create_dataloader, gen_id2corpus, gen_text_file
from ann_util import build_index
@@ -89,6 +92,15 @@
help="evaluate_result")
parser.add_argument('--evaluate', action='store_true',
help='whether evaluate while training')
+parser.add_argument("--use_amp", action="store_true", help="Whether to use AMP.")
+parser.add_argument("--amp_loss_scale", default=32768, type=float,help="The value of scale_loss for fp16. This is only used for AMP training.")
+parser.add_argument("--use_recompute",
+ action='store_true',
+ help="Using the recompute to scale up the batch size and save the memory.")
+parser.add_argument("--use_gradient_cache",
+ action='store_true',
+ help="Using the gradient cache to scale up the batch size and save the memory.")
+parser.add_argument("--chunk_numbers",type=int,default=50,help="The number of the chunks for model")
args = parser.parse_args()
# yapf: enable
@@ -161,6 +173,179 @@ def evaluate(model, corpus_data_loader, query_data_loader, recall_result_file,
return float(recall_N[1])
+def train(train_data_loader, model, optimizer, lr_scheduler, rank,
+ corpus_data_loader, query_data_loader, recall_result_file, text_list,
+ id2corpus, tokenizer):
+ global_step = 0
+ best_recall = 0.0
+ tic_train = time.time()
+ for epoch in range(1, args.epochs + 1):
+ for step, batch in enumerate(train_data_loader, start=1):
+ query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch
+
+ loss = model(query_input_ids=query_input_ids,
+ title_input_ids=title_input_ids,
+ query_token_type_ids=query_token_type_ids,
+ title_token_type_ids=title_token_type_ids)
+
+ global_step += 1
+ if global_step % args.log_steps == 0 and rank == 0:
+ print(
+ "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
+ % (global_step, epoch, step, loss, args.log_steps /
+ (time.time() - tic_train)))
+ tic_train = time.time()
+ loss.backward()
+ optimizer.step()
+ lr_scheduler.step()
+ optimizer.clear_grad()
+ if not args.evaluate:
+ if global_step % args.save_steps == 0 and rank == 0:
+ save_dir = os.path.join(args.save_dir,
+ "model_%d" % global_step)
+ if not os.path.exists(save_dir):
+ os.makedirs(save_dir)
+ save_param_path = os.path.join(save_dir,
+ 'model_state.pdparams')
+ paddle.save(model.state_dict(), save_param_path)
+ tokenizer.save_pretrained(save_dir)
+ if args.evaluate and rank == 0:
+ print("evaluating")
+ recall_5 = evaluate(model, corpus_data_loader, query_data_loader,
+ recall_result_file, text_list, id2corpus)
+ if recall_5 > best_recall:
+ best_recall = recall_5
+
+ save_dir = os.path.join(args.save_dir, "model_best")
+ if not os.path.exists(save_dir):
+ os.makedirs(save_dir)
+ save_param_path = os.path.join(save_dir, 'model_state.pdparams')
+ paddle.save(model.state_dict(), save_param_path)
+ tokenizer.save_pretrained(save_dir)
+ with open(os.path.join(save_dir, "train_result.txt"),
+ 'a',
+ encoding='utf-8') as fp:
+ fp.write('epoch=%d, global_step: %d, recall: %s\n' %
+ (epoch, global_step, recall_5))
+
+
+def gradient_cache_train(train_data_loader, model, optimizer, lr_scheduler,
+ rank, tokenizer):
+
+ if args.use_amp:
+ scaler = paddle.amp.GradScaler(init_loss_scaling=args.amp_loss_scale)
+
+ if args.batch_size % args.chunk_numbers == 0:
+ chunk_numbers = args.chunk_numbers
+ else:
+ raise Exception(
+ f" Batch_size {args.batch_size} must divides chunk_numbers {args.chunk_numbers} without producing a remainder "
+ )
+
+ def split(inputs, chunk_numbers, axis=0):
+ if inputs.shape[0] % chunk_numbers == 0:
+ return paddle.split(inputs, chunk_numbers, axis=0)
+ else:
+ return paddle.split(inputs, inputs.shape[0], axis=0)
+
+ global_step = 0
+ tic_train = time.time()
+ for epoch in range(1, args.epochs + 1):
+ for step, batch in enumerate(train_data_loader, start=1):
+ # Separate large batches into several sub batches
+ chunked_x = [split(t, chunk_numbers, axis=0) for t in batch]
+ sub_batchs = [list(s) for s in zip(*chunked_x)]
+
+ all_grads = []
+ all_CUDA_rnd_state = []
+ all_query = []
+ all_title = []
+
+ for sub_batch in sub_batchs:
+ all_reps = []
+ all_labels = []
+ sub_query_input_ids, sub_query_token_type_ids, sub_title_input_ids, sub_title_token_type_ids = sub_batch
+ with paddle.amp.auto_cast(
+ args.use_amp,
+ custom_white_list=["layer_norm", "softmax", "gelu"]):
+
+ with paddle.no_grad():
+ sub_CUDA_rnd_state = paddle.framework.random.get_cuda_rng_state(
+ )
+ all_CUDA_rnd_state.append(sub_CUDA_rnd_state)
+ sub_cosine_sim, sub_label, query_embedding, title_embedding = model(
+ query_input_ids=sub_query_input_ids,
+ title_input_ids=sub_title_input_ids,
+ query_token_type_ids=sub_query_token_type_ids,
+ title_token_type_ids=sub_title_token_type_ids)
+ all_reps.append(sub_cosine_sim)
+ all_labels.append(sub_label)
+ all_title.append(title_embedding)
+ all_query.append(query_embedding)
+
+ model_reps = paddle.concat(all_reps, axis=0)
+ model_title = paddle.concat(all_title)
+ model_query = paddle.concat(all_query)
+
+ model_title = model_title.detach()
+ model_query = model_query.detach()
+
+ model_query.stop_gradient = False
+ model_title.stop_gradient = False
+ model_reps.stop_gradient = False
+
+ model_label = paddle.concat(all_labels, axis=0)
+ loss = F.cross_entropy(input=model_reps, label=model_label)
+ loss.backward()
+ # Store gradients
+ all_grads.append(model_reps.grad)
+
+ for sub_batch, CUDA_state, grad in zip(sub_batchs,
+ all_CUDA_rnd_state,
+ all_grads):
+
+ sub_query_input_ids, sub_query_token_type_ids, sub_title_input_ids, sub_title_token_type_ids = sub_batch
+ paddle.framework.random.set_cuda_rng_state(CUDA_state)
+ # Recompute the forward propogation
+ sub_cosine_sim, sub_label, query_embedding, title_embedding = model(
+ query_input_ids=sub_query_input_ids,
+ title_input_ids=sub_title_input_ids,
+ query_token_type_ids=sub_query_token_type_ids,
+ title_token_type_ids=sub_title_token_type_ids)
+ # Chain rule
+ surrogate = paddle.dot(sub_cosine_sim, grad)
+ # Backward propogation
+ if args.use_amp:
+ scaled = scaler.scale(surrogate)
+ scaled.backward()
+ else:
+ surrogate.backward()
+ # Update model parameters
+ if args.use_amp:
+ scaler.minimize(optimizer, scaled)
+ else:
+ optimizer.step()
+
+ global_step += 1
+ if global_step % args.log_steps == 0 and rank == 0:
+ print(
+ "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
+ % (global_step, epoch, step, loss, args.log_steps /
+ (time.time() - tic_train)))
+ tic_train = time.time()
+
+ lr_scheduler.step()
+ optimizer.clear_grad()
+
+ if global_step % args.save_steps == 0 and rank == 0:
+ save_dir = os.path.join(args.save_dir, "model_%d" % global_step)
+ if not os.path.exists(save_dir):
+ os.makedirs(save_dir)
+ save_param_path = os.path.join(save_dir, 'model_state.pdparams')
+ paddle.save(model.state_dict(), save_param_path)
+ tokenizer.save_pretrained(save_dir)
+
+
def do_train():
paddle.set_device(args.device)
rank = paddle.distributed.get_rank()
@@ -173,7 +358,8 @@ def do_train():
data_path=args.train_set_file,
lazy=False)
- pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+ pretrained_model = AutoModel.from_pretrained(
+ args.model_name_or_path, enable_recompute=args.use_recompute)
tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
@@ -197,11 +383,16 @@ def do_train():
batch_size=args.batch_size,
batchify_fn=batchify_fn,
trans_fn=trans_func)
-
- model = SemanticIndexBatchNeg(pretrained_model,
- margin=args.margin,
- scale=args.scale,
- output_emb_size=args.output_emb_size)
+ if (args.use_gradient_cache):
+ model = SemanticIndexCacheNeg(pretrained_model,
+ margin=args.margin,
+ scale=args.scale,
+ output_emb_size=args.output_emb_size)
+ else:
+ model = SemanticIndexBatchNeg(pretrained_model,
+ margin=args.margin,
+ scale=args.scale,
+ output_emb_size=args.output_emb_size)
if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
state_dict = paddle.load(args.init_from_ckpt)
@@ -262,57 +453,13 @@ def do_train():
weight_decay=args.weight_decay,
apply_decay_param_fun=lambda x: x in decay_params)
- global_step = 0
- best_recall = 0.0
- tic_train = time.time()
- for epoch in range(1, args.epochs + 1):
- for step, batch in enumerate(train_data_loader, start=1):
- query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch
-
- loss = model(query_input_ids=query_input_ids,
- title_input_ids=title_input_ids,
- query_token_type_ids=query_token_type_ids,
- title_token_type_ids=title_token_type_ids)
-
- global_step += 1
- if global_step % args.log_steps == 0 and rank == 0:
- print(
- "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
- % (global_step, epoch, step, loss, 10 /
- (time.time() - tic_train)))
- tic_train = time.time()
- loss.backward()
- optimizer.step()
- lr_scheduler.step()
- optimizer.clear_grad()
- if not args.evaluate:
- if global_step % args.save_steps == 0 and rank == 0:
- save_dir = os.path.join(args.save_dir,
- "model_%d" % global_step)
- if not os.path.exists(save_dir):
- os.makedirs(save_dir)
- save_param_path = os.path.join(save_dir,
- 'model_state.pdparams')
- paddle.save(model.state_dict(), save_param_path)
- tokenizer.save_pretrained(save_dir)
- if args.evaluate and rank == 0:
- print("evaluating")
- recall_5 = evaluate(model, corpus_data_loader, query_data_loader,
- recall_result_file, text_list, id2corpus)
- if recall_5 > best_recall:
- best_recall = recall_5
-
- save_dir = os.path.join(args.save_dir, "model_best")
- if not os.path.exists(save_dir):
- os.makedirs(save_dir)
- save_param_path = os.path.join(save_dir, 'model_state.pdparams')
- paddle.save(model.state_dict(), save_param_path)
- tokenizer.save_pretrained(save_dir)
- with open(os.path.join(save_dir, "train_result.txt"),
- 'a',
- encoding='utf-8') as fp:
- fp.write('epoch=%d, global_step: %d, recall: %s\n' %
- (epoch, global_step, recall_5))
+ if (args.use_gradient_cache):
+ gradient_cache_train(train_data_loader, model, optimizer, lr_scheduler,
+ rank, tokenizer)
+ else:
+ train(train_data_loader, model, optimizer, lr_scheduler, rank,
+ corpus_data_loader, query_data_loader, recall_result_file,
+ text_list, id2corpus, tokenizer)
if __name__ == "__main__":
diff --git a/applications/text_classification/hierarchical/analysis/word_interpret.ipynb b/applications/text_classification/hierarchical/analysis/word_interpret.ipynb
index 8c25b87a2711..6ddad7051a32 100644
--- a/applications/text_classification/hierarchical/analysis/word_interpret.ipynb
+++ b/applications/text_classification/hierarchical/analysis/word_interpret.ipynb
@@ -112,6 +112,7 @@
" items = line.strip().split('\\t')\n",
" if items[0] == 'Text':\n",
" continue\n",
+ " items[0] = items[0][:MAX_LENGTH-2]\n",
" if len(items) == 3:\n",
" yield {'text': items[0], 'label': items[1], 'predict': items[2]}\n",
" elif len(items) == 2:\n",
diff --git a/applications/text_classification/hierarchical/deploy/predictor/README.md b/applications/text_classification/hierarchical/deploy/predictor/README.md
index caff6498386e..46102ba6d554 100644
--- a/applications/text_classification/hierarchical/deploy/predictor/README.md
+++ b/applications/text_classification/hierarchical/deploy/predictor/README.md
@@ -11,7 +11,7 @@
如果基于GPU部署,请先确保机器已正确安装NVIDIA相关驱动和基础软件,确保CUDA >= 11.2,CuDNN >= 8.2,并使用以下命令安装所需依赖:
```shell
-python -m pip install onnxruntime-gpu onnx onnxconverter-common psutil
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0 psutil
```
如果基于CPU部署,请使用如下命令安装所需依赖:
diff --git a/applications/text_classification/hierarchical/few-shot/README.md b/applications/text_classification/hierarchical/few-shot/README.md
index 347e522164f8..c1311eda6d1e 100644
--- a/applications/text_classification/hierarchical/few-shot/README.md
+++ b/applications/text_classification/hierarchical/few-shot/README.md
@@ -65,9 +65,9 @@
内存: 630 GB
-3. PaddlePaddle 版本:2.3.1
+3. PaddlePaddle 版本:2.4rc
-4. PaddleNLP 版本:2.3.5 (develop)
+4. PaddleNLP 版本:2.4.3
5. 评估设置
@@ -91,7 +91,7 @@
| model_name | 训练方式 | Micro F1分数 | Macro F1分数 |
| ---------- | ------- | ----------- | ----------- |
| ernie-3.0-base-zh | 微调学习 | 0.7172 | 0.3821 |
- | ernie-3.0-base-zh | 提示学习 | 0.8855 | 0.8443 |
+ | ernie-3.0-base-zh | 提示学习 | 0.8945 | 0.8516 |
@@ -102,10 +102,10 @@
### 3.1 运行环境
-- python >= 3.6
-- paddlepaddle > 2.3(2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html))
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
### 3.2 代码结构
@@ -222,12 +222,12 @@ python train.py \
--do_export \
--num_train_epochs 100 \
--logging_steps 5 \
+--save_total_limit 1 \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
--metric_for_best_model macro_f1_score \
--load_best_model_at_end \
---evaluation_strategy epoch \
---save_strategy epoch
+--eval_steps 100
```
**多卡训练**
@@ -247,12 +247,12 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
--do_export \
--num_train_epochs 100 \
--logging_steps 5 \
+--save_total_limit 1 \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
--metric_for_best_model macro_f1_score \
--load_best_model_at_end \
---evaluation_strategy epoch \
---save_strategy epoch
+--eval_steps 100
```
可配置参数说明:
@@ -273,6 +273,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
- `do_export`: 是否在运行结束时将模型导出为静态图,保存路径为`output_dir/export`。
- `num_train_epochs`: 训练的最大轮数。
- `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
- `device`: 使用的设备,默认为`gpu`。
- `eval_steps`: 评估模型的间隔步数。
- `logging_steps`: 打印日志的间隔步数。
@@ -352,9 +353,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
可配置参数说明:
- `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名,或者模型参数配置目录路径,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
- `data_dir`: 待推理数据所在路径,数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
- `batch_size`: 每次预测的样本数量。
- `device`: 选择推理设备,包括`cpu`和`gpu`。默认为`gpu`。
- `device_id`: 指定GPU设备ID。
diff --git a/applications/text_classification/hierarchical/few-shot/infer.py b/applications/text_classification/hierarchical/few-shot/infer.py
index 7442641d502f..5fc92a9dd933 100644
--- a/applications/text_classification/hierarchical/few-shot/infer.py
+++ b/applications/text_classification/hierarchical/few-shot/infer.py
@@ -14,23 +14,24 @@
import os
import six
+import json
import psutil
import argparse
import numpy as np
from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
import paddle2onnx
import onnxruntime as ort
# yapf: disable
parser = argparse.ArgumentParser()
parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,12 +104,6 @@ def __init__(self,
'device_id':
device_id
}])
- self.input_handles = [
- self.predictor.get_inputs()[0].name,
- self.predictor.get_inputs()[1].name,
- self.predictor.get_inputs()[2].name
- ]
-
if device == "gpu":
try:
assert 'CUDAExecutionProvider' in self.predictor.get_providers()
@@ -122,27 +117,52 @@ def __init__(self,
logger.info(">>> [InferBackend] Engine Created ...")
def infer(self, input_dict: dict):
- input_dict = {
- k: v
- for k, v in input_dict.items() if k in self.input_handles
- }
result = self.predictor.run(None, input_dict)
return result
class HierachicalPredictor(object):
- def __init__(self, args, label_list):
- self._label_list = label_list
- self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
- self._max_seq_length = args.max_seq_length
- self._batch_size = args.batch_size
- self.inference_backend = InferBackend(args.model_path_prefix,
- args.device, args.device_id,
- args.use_fp16, args.num_threads)
- self._template = AutoTemplate.load_from(
- os.path.dirname(args.model_path_prefix), self._tokenizer,
- args.max_seq_length)
+ def __init__(self, args):
+ self.args = args
+ self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+ self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+ self.template, self.labels, self.input_handles = self.post_init()
+ self.collate_fn = PromptDataCollatorWithPadding(
+ self.tokenizer,
+ padding=True,
+ return_tensors="np",
+ return_attention_mask=True)
+
+ self.inference_backend = InferBackend(self.args.model_path_prefix,
+ self.args.device,
+ self.args.device_id,
+ self.args.use_fp16,
+ self.args.num_threads)
+
+ def post_init(self):
+ export_path = os.path.dirname(self.args.model_path_prefix)
+ template_path = os.path.join(export_path, "template_config.json")
+ with open(template_path, "r") as fp:
+ prompt = json.load(fp)
+ template = AutoTemplate.create_from(prompt, self.tokenizer,
+ self.args.max_length,
+ self.model)
+ keywords = template.extract_template_keywords(template.prompt)
+ inputs = [
+ "input_ids", "token_type_ids", "position_ids", "attention_mask",
+ "masked_positions"
+ ]
+ if "soft" in keywords:
+ inputs.append("soft_token_ids")
+ if "encoder" in keywords:
+ inputs.append("encoder_ids")
+ verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+ with open(verbalizer_path, "r") as fp:
+ label_words = json.load(fp)
+ labels = sorted(list(label_words.keys()))
+
+ return template, labels, inputs
def predict(self, input_data: list):
encoded_inputs = self.preprocess(input_data)
@@ -155,14 +175,27 @@ def _infer(self, input_dict):
infer_data = self.inference_backend.infer(input_dict)
return infer_data
- def infer_batch(self, encoded_inputs):
- num_sample = len(encoded_inputs["input_ids"])
+ def infer_batch(self, inputs):
+ num_sample = len(inputs)
infer_data = None
num_infer_data = None
- for idx in range(0, num_sample, self._batch_size):
- l, r = idx, idx + self._batch_size
- keys = encoded_inputs.keys()
- input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+ for index in range(0, num_sample, self.args.batch_size):
+ left, right = index, index + self.args.batch_size
+ batch_dict = self.collate_fn(inputs[left:right])
+ input_dict = {}
+ for key in self.input_handles:
+ value = batch_dict[key]
+ if key == "attention_mask":
+ if value.ndim == 2:
+ value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+ elif value.ndim != 4:
+ raise ValueError(
+ "Expect attention mask with ndim=2 or 4, but get ndim={}"
+ .format(value.ndim))
+ value = value.astype("float32")
+ else:
+ value = value.astype("int64")
+ input_dict[key] = value
results = self._infer(input_dict)
if infer_data is None:
infer_data = [[x] for x in results]
@@ -175,16 +208,8 @@ def infer_batch(self, encoded_inputs):
return infer_data
def preprocess(self, input_data: list):
- text = [InputExample(text_a=x) for x in input_data]
- inputs = [self._template.wrap_one_example(x) for x in text]
- inputs = {
- "input_ids":
- np.array([x["input_ids"] for x in inputs], dtype="int64"),
- "mask_ids":
- np.array([x["mask_ids"] for x in inputs], dtype="int64"),
- "soft_token_ids":
- np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
- }
+ text = [{"text_a": x} for x in input_data]
+ inputs = [self.template(x) for x in text]
return inputs
@staticmethod
@@ -197,7 +222,7 @@ def postprocess(self, infer_data):
label_ids = np.argwhere(probs > threshold)
labels = [[] for _ in range(probs.shape[0])]
for idx, label_id in label_ids:
- labels[idx].append(self._label_list[label_id])
+ labels[idx].append(self.labels[label_id])
return {"label": labels}
def printer(self, result, input_data):
@@ -212,12 +237,10 @@ def printer(self, result, input_data):
for arg_name, arg_value in vars(args).items():
logger.info("{:20}: {}".format(arg_name, arg_value))
- export_path = os.path.dirname(args.model_path_prefix)
- labels, _ = Verbalizer.load_from(export_path)
+ predictor = HierachicalPredictor(args)
text_dir = os.path.join(args.data_dir, "data.txt")
with open(text_dir, "r", encoding="utf-8") as f:
text_list = [x.strip() for x in f.readlines()]
- predictor = HierachicalPredictor(args, labels)
predictor.predict(text_list)
diff --git a/applications/text_classification/hierarchical/few-shot/requirements_cpu.txt b/applications/text_classification/hierarchical/few-shot/requirements_cpu.txt
new file mode 100644
index 000000000000..bbe76e363f00
--- /dev/null
+++ b/applications/text_classification/hierarchical/few-shot/requirements_cpu.txt
@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime
diff --git a/applications/text_classification/hierarchical/few-shot/requirements_gpu.txt b/applications/text_classification/hierarchical/few-shot/requirements_gpu.txt
new file mode 100644
index 000000000000..66454bd8b6b5
--- /dev/null
+++ b/applications/text_classification/hierarchical/few-shot/requirements_gpu.txt
@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common
diff --git a/applications/text_classification/hierarchical/few-shot/train.py b/applications/text_classification/hierarchical/few-shot/train.py
index c7c0a2daa38a..aac82bff9ab5 100644
--- a/applications/text_classification/hierarchical/few-shot/train.py
+++ b/applications/text_classification/hierarchical/few-shot/train.py
@@ -15,6 +15,7 @@
from dataclasses import dataclass, field
import os
import sys
+from collections import defaultdict
import paddle
import paddle.nn.functional as F
@@ -41,9 +42,6 @@
class DataArguments:
data_dir: str = field(default="./data", metadata={"help": "The dataset dictionary includes train.txt, dev.txt, test.txt, label.txt and data.txt (optional) files."})
prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
- soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
- encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
-
@dataclass
class ModelArguments:
@@ -67,17 +65,20 @@ def main():
tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
# Define the template for preprocess and the verbalizer for postprocess.
- template = AutoTemplate.create_from(
- data_args.prompt,
- tokenizer,
- training_args.max_seq_length,
- model=model,
- prompt_encoder=data_args.soft_encoder,
- encoder_hidden_size=data_args.encoder_hidden_size)
- logger.info("Using template: {}".format(template.template))
+ template = AutoTemplate.create_from(data_args.prompt,
+ tokenizer,
+ training_args.max_seq_length,
+ model=model)
+ logger.info("Using template: {}".format(template.prompt))
label_file = os.path.join(data_args.data_dir, "label.txt")
- verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+ with open(label_file, "r", encoding="utf-8") as fp:
+ label_words = defaultdict(list)
+ for line in fp:
+ data = line.strip().split("==")
+ word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+ label_words[data[0]].append(word)
+ verbalizer = SoftVerbalizer(label_words, tokenizer, model)
# Load the few-shot datasets.
train_ds, dev_ds, test_ds = load_local_dataset(
@@ -139,11 +140,24 @@ def compute_metrics(eval_preds):
# Export static model.
if training_args.do_export:
+ template = prompt_model.template
+ template_keywords = template.extract_template_keywords(template.prompt)
input_spec = [
- InputSpec(shape=[None, None], dtype="int64"), # input_ids
- InputSpec(shape=[None, None], dtype="int64"), # mask_ids
- InputSpec(shape=[None, None], dtype="int64"), # soft_token_ids
+ InputSpec(shape=[None, None], dtype="int64"), # input_ids,
+ InputSpec(shape=[None, None], dtype="int64"), # token_type_ids
+ InputSpec(shape=[None, None], dtype="int64"), # position_ids
+ InputSpec(shape=[None, None, None, None],
+ dtype="float32") # attention_mask
]
+ if "mask" in template_keywords:
+ input_spec.append(InputSpec(shape=[None],
+ dtype="int64")) # masked_positions
+ if "soft" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # soft_token_ids
+ if "encoder" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # encoder_ids
export_path = os.path.join(training_args.output_dir, 'export')
trainer.export_model(export_path,
input_spec=input_spec,
diff --git a/applications/text_classification/hierarchical/few-shot/utils.py b/applications/text_classification/hierarchical/few-shot/utils.py
index 4880432310f3..e29aaa9db61d 100644
--- a/applications/text_classification/hierarchical/few-shot/utils.py
+++ b/applications/text_classification/hierarchical/few-shot/utils.py
@@ -15,7 +15,6 @@
import os
from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
def load_local_dataset(data_path, splits, label_list):
@@ -39,14 +38,14 @@ def _reader(data_file, label_list):
for idx, line in enumerate(fp):
data = line.strip().split("\t")
if len(data) == 1:
- yield InputExample(text_a=data[0])
+ yield {"text_a": data[0]}
else:
text, label = data
label = label.strip().split(",")
label = [
float(1) if x in label else float(0) for x in label_list
]
- yield InputExample(text_a=text, labels=label)
+ yield {"text_a": text, "labels": label}
split_map = {"train": "train.txt", "dev": "dev.txt", "test": "test.txt"}
datasets = []
diff --git a/applications/text_classification/multi_class/analysis/evaluate.py b/applications/text_classification/multi_class/analysis/evaluate.py
index 9aa272c650f9..ad716c21b048 100644
--- a/applications/text_classification/multi_class/analysis/evaluate.py
+++ b/applications/text_classification/multi_class/analysis/evaluate.py
@@ -156,16 +156,17 @@ def evaluate():
logger.info("Dev dataset size: {}".format(len(dev_ds)))
logger.info("Accuracy in dev dataset: {:.2f}%".format(report['accuracy'] *
100))
- logger.info("Top-2 accuracy in dev dataset: {:.2f}%".format(
- top_k_accuracy_score(y_true=labels,
- y_score=probs,
- k=2,
- labels=list(range(len(label_list)))) * 100))
- logger.info("Top-3 accuracy in dev dataset: {:.2f}%".format(
- top_k_accuracy_score(y_true=labels,
- y_score=probs,
- k=3,
- labels=list(range(len(label_list)))) * 100))
+ if len(labels) > 2:
+ logger.info("Top-2 accuracy in dev dataset: {:.2f}%".format(
+ top_k_accuracy_score(y_true=labels,
+ y_score=probs,
+ k=2,
+ labels=list(range(len(label_list)))) * 100))
+ logger.info("Top-3 accuracy in dev dataset: {:.2f}%".format(
+ top_k_accuracy_score(y_true=labels,
+ y_score=probs,
+ k=3,
+ labels=list(range(len(label_list)))) * 100))
for i, l in enumerate(label_list):
logger.info("Class name: {}".format(l))
diff --git a/applications/text_classification/multi_class/analysis/word_interpret.ipynb b/applications/text_classification/multi_class/analysis/word_interpret.ipynb
index 0b83eec1bdf5..ddfee4143137 100644
--- a/applications/text_classification/multi_class/analysis/word_interpret.ipynb
+++ b/applications/text_classification/multi_class/analysis/word_interpret.ipynb
@@ -112,6 +112,7 @@
" items = line.strip().split('\\t')\n",
" if items[0] == 'Text':\n",
" continue\n",
+ " items[0] = items[0][:MAX_LENGTH-2]\n",
" if len(items) == 3:\n",
" yield {'text': items[0], 'label': items[1], 'predict': items[2]}\n",
" elif len(items) == 2:\n",
diff --git a/applications/text_classification/multi_class/deploy/predictor/README.md b/applications/text_classification/multi_class/deploy/predictor/README.md
index 8959571cb6ab..5da5f714a025 100644
--- a/applications/text_classification/multi_class/deploy/predictor/README.md
+++ b/applications/text_classification/multi_class/deploy/predictor/README.md
@@ -12,7 +12,7 @@
如果基于GPU部署,请先确保机器已正确安装NVIDIA相关驱动和基础软件,确保CUDA >= 11.2,CuDNN >= 8.2,并使用以下命令安装所需依赖:
```
-python -m pip install onnxruntime-gpu onnx onnxconverter-common
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0
```
如果基于CPU部署,请使用如下命令安装所需依赖:
diff --git a/applications/text_classification/multi_class/few-shot/README.md b/applications/text_classification/multi_class/few-shot/README.md
index 3f6e5f759bbf..45f9b009bf34 100644
--- a/applications/text_classification/multi_class/few-shot/README.md
+++ b/applications/text_classification/multi_class/few-shot/README.md
@@ -65,9 +65,9 @@
内存: 630 GB
-3. PaddlePaddle 版本:2.3.1
+3. PaddlePaddle 版本:2.4rc
-4. PaddleNLP 版本:2.3.5 (develop)
+4. PaddleNLP 版本:2.4.3
5. 评估设置
@@ -82,7 +82,7 @@ python train.py --dataset_dir "./data/" --save_dir "./checkpoints" --max_seq_len
- 提示学习
```
-python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条新闻写的是" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128 --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model accuracy --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch
+python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条新闻写的是" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128 --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model accuracy --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch --save_total_limit 1
```
6. 精度评价指标:Accuracy
@@ -102,10 +102,10 @@ python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条
### 3.1 运行环境
-- python >= 3.6
-- paddlepaddle > 2.3 (2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html))
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
### 3.2 代码结构
@@ -204,14 +204,15 @@ python train.py \
--output_dir ./checkpoints/ \
--prompt "这条新闻标题的主题是" \
--max_seq_length 128 \
---learning_rate 3e-5 \
---ppt_learning_rate 3e-4 \
+--learning_rate 3e-6 \
+--ppt_learning_rate 3e-5 \
--do_train \
--do_eval \
--use_rdrop \
--max_steps 1000 \
--eval_steps 10 \
--logging_steps 5 \
+--save_total_limit 1 \
--load_best_model_at_end True \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
@@ -227,8 +228,8 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
--output_dir ./checkpoints/ \
--prompt "这条新闻标题的主题是" \
--max_seq_length 128 \
---learning_rate 3e-5 \
---ppt_learning_rate 3e-4 \
+--learning_rate 3e-6 \
+--ppt_learning_rate 3e-5 \
--do_train \
--do_eval \
--use_rdrop \
@@ -236,6 +237,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
--max_steps 1000 \
--eval_steps 10 \
--logging_steps 5 \
+--save_total_limit 1 \
--load_best_model_at_end True \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
@@ -260,6 +262,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
- `do_predict`: 是否进行预测。
- `do_export`: 是否在运行结束时将模型导出为静态图,保存路径为`output_dir/export`。
- `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
- `eval_steps`: 评估模型的间隔步数。
- `device`: 使用的设备,默认为`gpu`。
- `logging_steps`: 打印日志的间隔步数。
@@ -335,9 +338,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
可配置参数说明:
- `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名,或者模型参数配置目录路径,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
- `data_dir`: 待推理数据所在路径,数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
- `batch_size`: 每次预测的样本数量。
- `device`: 选择推理设备,包括`cpu`和`gpu`。默认为`gpu`。
- `device_id`: 指定GPU设备ID。
diff --git a/applications/text_classification/multi_class/few-shot/infer.py b/applications/text_classification/multi_class/few-shot/infer.py
index 142a20d1ed76..03715bc29d0e 100644
--- a/applications/text_classification/multi_class/few-shot/infer.py
+++ b/applications/text_classification/multi_class/few-shot/infer.py
@@ -14,23 +14,24 @@
import os
import six
+import json
import psutil
import argparse
import numpy as np
from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
import paddle2onnx
import onnxruntime as ort
# yapf: disable
parser = argparse.ArgumentParser()
parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,11 +104,6 @@ def __init__(self,
'device_id':
device_id
}])
- self.input_handles = [
- self.predictor.get_inputs()[0].name,
- self.predictor.get_inputs()[1].name,
- self.predictor.get_inputs()[2].name
- ]
if device == "gpu":
try:
@@ -122,27 +118,53 @@ def __init__(self,
logger.info(">>> [InferBackend] Engine Created ...")
def infer(self, input_dict: dict):
- input_dict = {
- k: v
- for k, v in input_dict.items() if k in self.input_handles
- }
result = self.predictor.run(None, input_dict)
return result
class MultiClassPredictor(object):
- def __init__(self, args, label_list):
- self._label_list = label_list
- self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
- self._max_seq_length = args.max_seq_length
- self._batch_size = args.batch_size
- self.inference_backend = InferBackend(args.model_path_prefix,
- args.device, args.device_id,
- args.use_fp16, args.num_threads)
- self._template = AutoTemplate.load_from(
- os.path.dirname(args.model_path_prefix), self._tokenizer,
- args.max_seq_length)
+ def __init__(self, args):
+ self.args = args
+ self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+ self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+ self.template, self.labels, self.input_handles = self.post_init()
+ self.collate_fn = PromptDataCollatorWithPadding(
+ self.tokenizer,
+ padding=True,
+ return_tensors="np",
+ return_attention_mask=True)
+
+ self.inference_backend = InferBackend(self.args.model_path_prefix,
+ self.args.device,
+ self.args.device_id,
+ self.args.use_fp16,
+ self.args.num_threads)
+
+ def post_init(self):
+ export_path = os.path.dirname(self.args.model_path_prefix)
+ template_path = os.path.join(export_path, "template_config.json")
+ with open(template_path, "r") as fp:
+ prompt = json.load(fp)
+ template = AutoTemplate.create_from(prompt, self.tokenizer,
+ self.args.max_length,
+ self.model)
+ keywords = template.extract_template_keywords(template.prompt)
+ inputs = [
+ "input_ids", "token_type_ids", "position_ids", "attention_mask"
+ ]
+ if "mask" in keywords:
+ inputs.append("masked_positions")
+ if "soft" in keywords:
+ inputs.append("soft_token_ids")
+ if "encoder" in keywords:
+ inputs.append("encoder_ids")
+ verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+ with open(verbalizer_path, "r") as fp:
+ label_words = json.load(fp)
+ labels = sorted(list(label_words.keys()))
+
+ return template, labels, inputs
def predict(self, input_data: list):
encoded_inputs = self.preprocess(input_data)
@@ -155,14 +177,27 @@ def _infer(self, input_dict):
infer_data = self.inference_backend.infer(input_dict)
return infer_data
- def infer_batch(self, encoded_inputs):
- num_sample = len(encoded_inputs["input_ids"])
+ def infer_batch(self, inputs):
+ num_sample = len(inputs)
infer_data = None
num_infer_data = None
- for idx in range(0, num_sample, self._batch_size):
- l, r = idx, idx + self._batch_size
- keys = encoded_inputs.keys()
- input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+ for index in range(0, num_sample, self.args.batch_size):
+ left, right = index, index + self.args.batch_size
+ batch_dict = self.collate_fn(inputs[left:right])
+ input_dict = {}
+ for key in self.input_handles:
+ value = batch_dict[key]
+ if key == "attention_mask":
+ if value.ndim == 2:
+ value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+ elif value.ndim != 4:
+ raise ValueError(
+ "Expect attention mask with ndim=2 or 4, but get ndim={}"
+ .format(value.ndim))
+ value = value.astype("float32")
+ else:
+ value = value.astype("int64")
+ input_dict[key] = value
results = self._infer(input_dict)
if infer_data is None:
infer_data = [[x] for x in results]
@@ -175,21 +210,13 @@ def infer_batch(self, encoded_inputs):
return infer_data
def preprocess(self, input_data: list):
- text = [InputExample(text_a=x) for x in input_data]
- inputs = [self._template.wrap_one_example(x) for x in text]
- inputs = {
- "input_ids":
- np.array([x["input_ids"] for x in inputs], dtype="int64"),
- "mask_ids":
- np.array([x["mask_ids"] for x in inputs], dtype="int64"),
- "soft_token_ids":
- np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
- }
+ text = [{"text_a": x} for x in input_data]
+ inputs = [self.template(x) for x in text]
return inputs
def postprocess(self, infer_data):
preds = np.argmax(infer_data[0], axis=-1)
- labels = [self._label_list[x] for x in preds]
+ labels = [self.labels[x] for x in preds]
return {"label": labels}
def printer(self, result, input_data):
@@ -204,12 +231,10 @@ def printer(self, result, input_data):
for arg_name, arg_value in vars(args).items():
logger.info("{:20}: {}".format(arg_name, arg_value))
- export_path = os.path.dirname(args.model_path_prefix)
- labels, _ = Verbalizer.load_from(export_path)
+ predictor = MultiClassPredictor(args)
text_dir = os.path.join(args.data_dir, "data.txt")
with open(text_dir, "r", encoding="utf-8") as f:
text_list = [x.strip() for x in f.readlines()]
- predictor = MultiClassPredictor(args, labels)
predictor.predict(text_list)
diff --git a/applications/text_classification/multi_class/few-shot/requirements_cpu.txt b/applications/text_classification/multi_class/few-shot/requirements_cpu.txt
new file mode 100644
index 000000000000..bbe76e363f00
--- /dev/null
+++ b/applications/text_classification/multi_class/few-shot/requirements_cpu.txt
@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime
diff --git a/applications/text_classification/multi_class/few-shot/requirements_gpu.txt b/applications/text_classification/multi_class/few-shot/requirements_gpu.txt
new file mode 100644
index 000000000000..66454bd8b6b5
--- /dev/null
+++ b/applications/text_classification/multi_class/few-shot/requirements_gpu.txt
@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common
diff --git a/applications/text_classification/multi_class/few-shot/train.py b/applications/text_classification/multi_class/few-shot/train.py
index a07d27ea5879..0e67750deb23 100644
--- a/applications/text_classification/multi_class/few-shot/train.py
+++ b/applications/text_classification/multi_class/few-shot/train.py
@@ -14,6 +14,9 @@
from dataclasses import dataclass, field
import os
+from collections import defaultdict
+
+import numpy as np
import paddle
from paddle.static import InputSpec
@@ -37,8 +40,6 @@
class DataArguments:
data_dir: str = field(default="./data/", metadata={"help": "Path to a dataset which includes train.txt, dev.txt, test.txt, label.txt and data.txt (optional)."})
prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
- soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
- encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
@dataclass
@@ -63,17 +64,20 @@ def main():
tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
# Define the template for preprocess and the verbalizer for postprocess.
- template = AutoTemplate.create_from(
- data_args.prompt,
- tokenizer,
- training_args.max_seq_length,
- model=model,
- prompt_encoder=data_args.soft_encoder,
- encoder_hidden_size=data_args.encoder_hidden_size)
- logger.info("Using template: {}".format(template.template))
+ template = AutoTemplate.create_from(data_args.prompt,
+ tokenizer,
+ training_args.max_seq_length,
+ model=model)
+ logger.info("Using template: {}".format(template.prompt))
label_file = os.path.join(data_args.data_dir, "label.txt")
- verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+ with open(label_file, "r", encoding="utf-8") as fp:
+ label_words = defaultdict(list)
+ for line in fp:
+ data = line.strip().split("==")
+ word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+ label_words[data[0]].append(word)
+ verbalizer = SoftVerbalizer(label_words, tokenizer, model)
# Load the few-shot datasets.
train_ds, dev_ds, test_ds = load_local_dataset(
@@ -133,11 +137,24 @@ def compute_metrics(eval_preds):
# Export static model.
if training_args.do_export:
+ template = prompt_model.template
+ template_keywords = template.extract_template_keywords(template.prompt)
input_spec = [
- InputSpec(shape=[None, None], dtype="int64"), # input_ids
- InputSpec(shape=[None, None], dtype="int64"), # mask_ids
- InputSpec(shape=[None, None], dtype="int64"), # soft_token_ids
+ InputSpec(shape=[None, None], dtype="int64"), # input_ids,
+ InputSpec(shape=[None, None], dtype="int64"), # token_type_ids
+ InputSpec(shape=[None, None], dtype="int64"), # position_ids
+ InputSpec(shape=[None, None, None, None],
+ dtype="float32") # attention_mask
]
+ if "mask" in template_keywords:
+ input_spec.append(InputSpec(shape=[None],
+ dtype="int64")) # masked_positions
+ if "soft" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # soft_token_ids
+ if "encoder" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # encoder_ids
export_path = os.path.join(training_args.output_dir, 'export')
trainer.export_model(export_path,
input_spec=input_spec,
diff --git a/applications/text_classification/multi_class/few-shot/utils.py b/applications/text_classification/multi_class/few-shot/utils.py
index 907b640ea96b..b440c9627917 100644
--- a/applications/text_classification/multi_class/few-shot/utils.py
+++ b/applications/text_classification/multi_class/few-shot/utils.py
@@ -15,7 +15,6 @@
import os
from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
def load_local_dataset(data_path, splits, label_list):
@@ -38,10 +37,10 @@ def _reader(data_file, label_list):
for idx, line in enumerate(fp):
data = line.strip().split("\t")
if len(data) == 1:
- yield InputExample(text_a=data[0])
+ yield {"text_a": data[0]}
else:
text, label = data
- yield InputExample(text_a=text, labels=label_list[label])
+ yield {"text_a": text, "labels": label_list[label]}
assert isinstance(splits, list) and len(splits) > 0
diff --git a/applications/text_classification/multi_label/analysis/word_interpret.ipynb b/applications/text_classification/multi_label/analysis/word_interpret.ipynb
index 07dd3f85c839..d6601cd4d4ed 100644
--- a/applications/text_classification/multi_label/analysis/word_interpret.ipynb
+++ b/applications/text_classification/multi_label/analysis/word_interpret.ipynb
@@ -1,32 +1,8 @@
{
- "metadata": {
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.7.13-final"
- },
- "orig_nbformat": 2,
- "kernelspec": {
- "name": "python3",
- "display_name": "Python 3.7.13 64-bit",
- "metadata": {
- "interpreter": {
- "hash": "767d51c1340bd893661ea55ea3124f6de3c7a262a8b4abca0554b478b1e2ff90"
- }
- }
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2,
"cells": [
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"# 词级别可解释性分析\n",
"本项目提供模型的词级别可解释性分析,包括LIME、Integrated Gradient、GradShap 三种分析方法,支持分析微调后模型的预测结果,开发者可以通过更改**数据目录**和**模型目录**在自己的任务中使用此项目进行数据分析。\n",
@@ -54,34 +30,16 @@
"准予原告胡某甲与被告韩某甲离婚。\n",
"...\n",
"```\n"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
- "source": [
- "import functools\n",
- "import random\n",
- "import os\n",
- "import argparse\n",
- "\n",
- "import jieba\n",
- "import numpy as np\n",
- "from trustai.interpretation import VisualizationTextRecord\n",
- "from trustai.interpretation import get_word_offset\n",
- "import paddle\n",
- "from paddle.io import DataLoader, BatchSampler\n",
- "from paddlenlp.data import DataCollatorWithPadding\n",
- "from paddlenlp.datasets import load_dataset\n",
- "from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer\n"
- ],
"cell_type": "code",
- "metadata": {},
"execution_count": 1,
+ "metadata": {},
"outputs": [
{
- "output_type": "stream",
"name": "stderr",
+ "output_type": "stream",
"text": [
"grep: warning: GREP_OPTIONS is deprecated; please use an alias or script\n",
"/usr/local/lib/python3.7/dist-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
@@ -96,6 +54,22 @@
" resample=Image.BICUBIC,\n"
]
}
+ ],
+ "source": [
+ "import functools\n",
+ "import random\n",
+ "import os\n",
+ "import argparse\n",
+ "\n",
+ "import jieba\n",
+ "import numpy as np\n",
+ "from trustai.interpretation import VisualizationTextRecord\n",
+ "from trustai.interpretation import get_word_offset\n",
+ "import paddle\n",
+ "from paddle.io import DataLoader, BatchSampler\n",
+ "from paddlenlp.data import DataCollatorWithPadding\n",
+ "from paddlenlp.datasets import load_dataset\n",
+ "from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer\n"
]
},
{
@@ -127,11 +101,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"## 2.读取待分析数据"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -148,6 +122,7 @@
" items = line.strip().split('\\t')\n",
" if items[0] == 'Text':\n",
" continue\n",
+ " items[0] = items[0][:MAX_LENGTH-2]\n",
" if len(items) == 3:\n",
" yield {'text': items[0], 'label': items[1], 'predict': items[2]}\n",
" elif len(items) == 2:\n",
@@ -181,8 +156,8 @@
"metadata": {},
"outputs": [
{
- "output_type": "stream",
"name": "stderr",
+ "output_type": "stream",
"text": [
"\u001b[32m[2022-09-28 04:51:03,566] [ INFO]\u001b[0m - We are using
to load '../checkpoint/'.\u001b[0m\n",
"W0928 04:51:03.570216 4827 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2\n",
@@ -229,12 +204,12 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"## 3.开始数据可解释性分析\n",
"数据量较大时,数据分析时间较长,请耐心等待"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -242,8 +217,8 @@
"metadata": {},
"outputs": [
{
- "output_type": "stream",
"name": "stdout",
+ "output_type": "stream",
"text": [
"Start token level interpretion, it will take some time...\n",
"Building prefix dict from the default dictionary ...\n",
@@ -292,12 +267,12 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"## 4.数据可解释性分析结果可视化\n",
"使用用颜色深浅可视化方式代表句子中词对预测结果的重要程度"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -364,12 +339,16 @@
"metadata": {},
"outputs": [
{
- "output_type": "display_data",
"data": {
- "text/plain": "",
- "text/html": "Label | Prediction | Key words | Important visualization |
---|
不履行家庭义务,婚后分居 | 婚后分居 | 至今 双方 出 分居 。 | [CLS] 2015 年 2 月 23 日 , 被告 将 原告 赶 出 家门 , 原告 居住 于 娘家 待产 , 双方 分居 至今 。 [SEP] |
|
婚后有子女,限制行为能力子女抚养 | 婚后有子女,限制行为能力子女抚养,不履行离婚协议 | 财产 符合 付清 欠条 抚养 | [CLS] 被告 孙某 辩称 : 离婚 协议 中 关于 财产 分割 给付 资金 不 符合 法律 规定 , 只有 离婚 和 子女 抚养 符合 法律 规定 , 就 没有 么 , 协议 不 代表 被告 的 真实 意思 表示 , 离婚 协议 中 没有 约定 付款 时间 , 而且 被告 给 原告 出具 了 欠条 是 5 年内 付清 , 原告 未 在 期满 后 起诉 , 应 驳回 。 [SEP] |
|
存在非婚生子,支付抚养费,限制行为能力子女抚养 | 限制行为能力子女抚养,存在非婚生子 | 赵某 并非 认可 之女 表示 | [CLS] 被告 董某 认可 赵某 乙 并非 原告 之女 , 并 表示 愿意 自行 抚养 赵某 乙 。 [SEP] |
|
准予离婚 | 准予离婚,法定离婚 | 原告 韩某 准予 离婚 。 | [CLS] 准予 原告 胡某 甲 与 被告 韩某 甲 离婚 。 [SEP] |
|
"
+ "text/html": [
+ "Label | Prediction | Key words | Important visualization |
---|
不履行家庭义务,婚后分居 | 婚后分居 | 至今 双方 出 分居 。 | [CLS] 2015 年 2 月 23 日 , 被告 将 原告 赶 出 家门 , 原告 居住 于 娘家 待产 , 双方 分居 至今 。 [SEP] |
|
婚后有子女,限制行为能力子女抚养 | 婚后有子女,限制行为能力子女抚养,不履行离婚协议 | 财产 符合 付清 欠条 抚养 | [CLS] 被告 孙某 辩称 : 离婚 协议 中 关于 财产 分割 给付 资金 不 符合 法律 规定 , 只有 离婚 和 子女 抚养 符合 法律 规定 , 就 没有 么 , 协议 不 代表 被告 的 真实 意思 表示 , 离婚 协议 中 没有 约定 付款 时间 , 而且 被告 给 原告 出具 了 欠条 是 5 年内 付清 , 原告 未 在 期满 后 起诉 , 应 驳回 。 [SEP] |
|
存在非婚生子,支付抚养费,限制行为能力子女抚养 | 限制行为能力子女抚养,存在非婚生子 | 赵某 并非 认可 之女 表示 | [CLS] 被告 董某 认可 赵某 乙 并非 原告 之女 , 并 表示 愿意 自行 抚养 赵某 乙 。 [SEP] |
|
准予离婚 | 准予离婚,法定离婚 | 原告 韩某 准予 离婚 。 | [CLS] 准予 原告 胡某 甲 与 被告 韩某 甲 离婚 。 [SEP] |
|
"
+ ],
+ "text/plain": [
+ ""
+ ]
},
- "metadata": {}
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
@@ -377,5 +356,31 @@
"html = visualize(align_res, interpret_ds)"
]
}
- ]
-}
\ No newline at end of file
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3.7.13 64-bit",
+ "metadata": {
+ "interpreter": {
+ "hash": "767d51c1340bd893661ea55ea3124f6de3c7a262a8b4abca0554b478b1e2ff90"
+ }
+ },
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13-final"
+ },
+ "orig_nbformat": 2
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/applications/text_classification/multi_label/deploy/predictor/README.md b/applications/text_classification/multi_label/deploy/predictor/README.md
index 4c7ff45c8aab..fb9aca5a731e 100644
--- a/applications/text_classification/multi_label/deploy/predictor/README.md
+++ b/applications/text_classification/multi_label/deploy/predictor/README.md
@@ -12,7 +12,7 @@
如果基于GPU部署,请先确保机器已正确安装NVIDIA相关驱动和基础软件,确保CUDA >= 11.2,CuDNN >= 8.2,并使用以下命令安装所需依赖:
```shell
-python -m pip install onnxruntime-gpu onnx onnxconverter-common
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0
```
如果基于CPU部署,请使用如下命令安装所需依赖:
diff --git a/applications/text_classification/multi_label/few-shot/README.md b/applications/text_classification/multi_label/few-shot/README.md
index d7ee94ff68f2..4bb3bbf51fb5 100644
--- a/applications/text_classification/multi_label/few-shot/README.md
+++ b/applications/text_classification/multi_label/few-shot/README.md
@@ -70,9 +70,9 @@
内存: 630 GB
-3. PaddlePaddle 版本:2.3.1
+3. PaddlePaddle 版本:2.4rc
-4. PaddleNLP 版本:2.3.5 (develop)
+4. PaddleNLP 版本:2.4.3
5. 评估设置
@@ -88,7 +88,7 @@
- 提示学习
```
- python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这句话包含的要素有" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128 --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model macro_f1_score --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch
+ python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这句话包含的要素有" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128 --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model macro_f1_score --load_best_model_at_end --eval_steps 100 --save_total_limit 1
```
6. 精度评价指标:Micro F1分数、Macro F1分数
@@ -96,7 +96,7 @@
| model_name | 训练方式 | Micro F1分数 | Macro F1分数 |
| ---------- | ------- | ----------- | ----------- |
| ernie-3.0-base-zh | 微调学习 | 0.7419 | 0.5105 |
- | ernie-3.0-base-zh | 提示学习 | 0.7838 | 0.6985 |
+ | ernie-3.0-base-zh | 提示学习 | 0.7839 | 0.6003 |
## 3.定制训练
@@ -106,10 +106,10 @@
### 3.1 运行环境
-- python >= 3.6
-- paddlepaddle > 2.3(2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html))
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
### 3.2 代码结构
@@ -223,6 +223,7 @@ python train.py \
--do_export \
--num_train_epochs 100 \
--logging_steps 5 \
+--save_total_limit 1 \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
--metric_for_best_model macro_f1_score \
@@ -248,6 +249,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
--do_export \
--num_train_epochs 100 \
--logging_steps 5 \
+--save_total_limit 1 \
--per_device_eval_batch_size 32 \
--per_device_train_batch_size 8 \
--metric_for_best_model macro_f1_score \
@@ -274,6 +276,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
- `do_export`: 是否在运行结束时将模型导出为静态图,保存路径为`output_dir/export`。
- `num_train_epochs`: 训练的最大轮数。
- `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
- `device`: 使用的设备,默认为`gpu`。
- `eval_steps`: 评估模型的间隔步数。
- `logging_steps`: 打印日志的间隔步数。
@@ -352,9 +355,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
可配置参数说明:
- `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名,或者模型参数配置目录路径,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名,用于加载tokenizer。默认为`ernie-3.0-base-zh`。
- `data_dir`: 待推理数据所在路径,数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度,超过该长度的文本将被截断,不足的以Pad补全。提示文本不会被截断。
- `batch_size`: 每次预测的样本数量。
- `device`: 选择推理设备,包括`cpu`和`gpu`。默认为`gpu`。
- `device_id`: 指定GPU设备ID。
diff --git a/applications/text_classification/multi_label/few-shot/infer.py b/applications/text_classification/multi_label/few-shot/infer.py
index 48d42d294e96..2014afaef56b 100644
--- a/applications/text_classification/multi_label/few-shot/infer.py
+++ b/applications/text_classification/multi_label/few-shot/infer.py
@@ -14,23 +14,24 @@
import os
import six
+import json
import psutil
import argparse
import numpy as np
from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
import paddle2onnx
import onnxruntime as ort
# yapf: disable
parser = argparse.ArgumentParser()
parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,11 +104,6 @@ def __init__(self,
'device_id':
device_id
}])
- self.input_handles = [
- self.predictor.get_inputs()[0].name,
- self.predictor.get_inputs()[1].name,
- self.predictor.get_inputs()[2].name
- ]
if device == "gpu":
try:
@@ -122,27 +118,52 @@ def __init__(self,
logger.info(">>> [InferBackend] Engine Created ...")
def infer(self, input_dict: dict):
- input_dict = {
- k: v
- for k, v in input_dict.items() if k in self.input_handles
- }
result = self.predictor.run(None, input_dict)
return result
class MultiLabelPredictor(object):
- def __init__(self, args, label_list):
- self._label_list = label_list
- self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
- self._max_seq_length = args.max_seq_length
- self._batch_size = args.batch_size
- self.inference_backend = InferBackend(args.model_path_prefix,
- args.device, args.device_id,
- args.use_fp16, args.num_threads)
- self._template = AutoTemplate.load_from(
- os.path.dirname(args.model_path_prefix), self._tokenizer,
- args.max_seq_length)
+ def __init__(self, args):
+ self.args = args
+ self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+ self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+ self.template, self.labels, self.input_handles = self.post_init()
+ self.collate_fn = PromptDataCollatorWithPadding(
+ self.tokenizer,
+ padding=True,
+ return_tensors="np",
+ return_attention_mask=True)
+
+ self.inference_backend = InferBackend(self.args.model_path_prefix,
+ self.args.device,
+ self.args.device_id,
+ self.args.use_fp16,
+ self.args.num_threads)
+
+ def post_init(self):
+ export_path = os.path.dirname(self.args.model_path_prefix)
+ template_path = os.path.join(export_path, "template_config.json")
+ with open(template_path, "r") as fp:
+ prompt = json.load(fp)
+ template = AutoTemplate.create_from(prompt, self.tokenizer,
+ self.args.max_length,
+ self.model)
+ keywords = template.extract_template_keywords(template.prompt)
+ inputs = [
+ "input_ids", "token_type_ids", "position_ids", "attention_mask",
+ "masked_positions"
+ ]
+ if "soft" in keywords:
+ inputs.append("soft_token_ids")
+ if "encoder" in keywords:
+ inputs.append("encoder_ids")
+ verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+ with open(verbalizer_path, "r") as fp:
+ label_words = json.load(fp)
+ labels = sorted(list(label_words.keys()))
+
+ return template, labels, inputs
def predict(self, input_data: list):
encoded_inputs = self.preprocess(input_data)
@@ -155,14 +176,27 @@ def _infer(self, input_dict):
infer_data = self.inference_backend.infer(input_dict)
return infer_data
- def infer_batch(self, encoded_inputs):
- num_sample = len(encoded_inputs["input_ids"])
+ def infer_batch(self, inputs):
+ num_sample = len(inputs)
infer_data = None
num_infer_data = None
- for idx in range(0, num_sample, self._batch_size):
- l, r = idx, idx + self._batch_size
- keys = encoded_inputs.keys()
- input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+ for index in range(0, num_sample, self.args.batch_size):
+ left, right = index, index + self.args.batch_size
+ batch_dict = self.collate_fn(inputs[left:right])
+ input_dict = {}
+ for key in self.input_handles:
+ value = batch_dict[key]
+ if key == "attention_mask":
+ if value.ndim == 2:
+ value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+ elif value.ndim != 4:
+ raise ValueError(
+ "Expect attention mask with ndim=2 or 4, but get ndim={}"
+ .format(value.ndim))
+ value = value.astype("float32")
+ else:
+ value = value.astype("int64")
+ input_dict[key] = value
results = self._infer(input_dict)
if infer_data is None:
infer_data = [[x] for x in results]
@@ -175,16 +209,8 @@ def infer_batch(self, encoded_inputs):
return infer_data
def preprocess(self, input_data: list):
- text = [InputExample(text_a=x) for x in input_data]
- inputs = [self._template.wrap_one_example(x) for x in text]
- inputs = {
- "input_ids":
- np.array([x["input_ids"] for x in inputs], dtype="int64"),
- "mask_ids":
- np.array([x["mask_ids"] for x in inputs], dtype="int64"),
- "soft_token_ids":
- np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
- }
+ text = [{"text_a": x} for x in input_data]
+ inputs = [self.template(x) for x in text]
return inputs
@staticmethod
@@ -197,7 +223,7 @@ def postprocess(self, infer_data):
label_ids = np.argwhere(probs > threshold)
labels = [[] for _ in range(probs.shape[0])]
for idx, label_id in label_ids:
- labels[idx].append(self._label_list[label_id])
+ labels[idx].append(self.labels[label_id])
return {"label": labels}
def printer(self, result, input_data):
@@ -212,12 +238,10 @@ def printer(self, result, input_data):
for arg_name, arg_value in vars(args).items():
logger.info("{:20}: {}".format(arg_name, arg_value))
- export_path = os.path.dirname(args.model_path_prefix)
- labels, _ = Verbalizer.load_from(export_path)
+ predictor = MultiLabelPredictor(args)
text_dir = os.path.join(args.data_dir, "data.txt")
with open(text_dir, "r", encoding="utf-8") as f:
text_list = [x.strip() for x in f.readlines()]
- predictor = MultiLabelPredictor(args, labels)
predictor.predict(text_list)
diff --git a/applications/text_classification/multi_label/few-shot/requirements_cpu.txt b/applications/text_classification/multi_label/few-shot/requirements_cpu.txt
new file mode 100644
index 000000000000..bbe76e363f00
--- /dev/null
+++ b/applications/text_classification/multi_label/few-shot/requirements_cpu.txt
@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime
diff --git a/applications/text_classification/multi_label/few-shot/requirements_gpu.txt b/applications/text_classification/multi_label/few-shot/requirements_gpu.txt
new file mode 100644
index 000000000000..66454bd8b6b5
--- /dev/null
+++ b/applications/text_classification/multi_label/few-shot/requirements_gpu.txt
@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common
diff --git a/applications/text_classification/multi_label/few-shot/train.py b/applications/text_classification/multi_label/few-shot/train.py
index 44f0c190c90f..6d12f6b3e5fd 100644
--- a/applications/text_classification/multi_label/few-shot/train.py
+++ b/applications/text_classification/multi_label/few-shot/train.py
@@ -15,6 +15,7 @@
from dataclasses import dataclass, field
import os
import sys
+from collections import defaultdict
import paddle
import paddle.nn.functional as F
@@ -41,8 +42,6 @@
class DataArguments:
data_dir: str = field(default="./data", metadata={"help": "The dataset dictionary includes train.txt, dev.txt and label.txt files."})
prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
- soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
- encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
@dataclass
@@ -67,17 +66,20 @@ def main():
tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
# Define the template for preprocess and the verbalizer for postprocess.
- template = AutoTemplate.create_from(
- data_args.prompt,
- tokenizer,
- training_args.max_seq_length,
- model=model,
- prompt_encoder=data_args.soft_encoder,
- encoder_hidden_size=data_args.encoder_hidden_size)
- logger.info("Using template: {}".format(template.template))
+ template = AutoTemplate.create_from(data_args.prompt,
+ tokenizer,
+ training_args.max_seq_length,
+ model=model)
+ logger.info("Using template: {}".format(template.prompt))
label_file = os.path.join(data_args.data_dir, "label.txt")
- verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+ with open(label_file, "r", encoding="utf-8") as fp:
+ label_words = defaultdict(list)
+ for line in fp:
+ data = line.strip().split("==")
+ word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+ label_words[data[0]].append(word)
+ verbalizer = SoftVerbalizer(label_words, tokenizer, model)
# Load the few-shot datasets.
train_ds, dev_ds, test_ds = load_local_dataset(
@@ -139,11 +141,24 @@ def compute_metrics(eval_preds):
# Export static model.
if training_args.do_export:
+ template = prompt_model.template
+ template_keywords = template.extract_template_keywords(template.prompt)
input_spec = [
- InputSpec(shape=[None, None], dtype="int64"), # input_ids
- InputSpec(shape=[None, None], dtype="int64"), # mask_ids
- InputSpec(shape=[None, None], dtype="int64"), # soft_token_ids
+ InputSpec(shape=[None, None], dtype="int64"), # input_ids,
+ InputSpec(shape=[None, None], dtype="int64"), # token_type_ids
+ InputSpec(shape=[None, None], dtype="int64"), # position_ids
+ InputSpec(shape=[None, None, None, None],
+ dtype="float32") # attention_mask
]
+ if "mask" in template_keywords:
+ input_spec.append(InputSpec(shape=[None],
+ dtype="int64")) # masked_positions
+ if "soft" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # soft_token_ids
+ if "encoder" in template_keywords:
+ input_spec.append(InputSpec(shape=[None, None],
+ dtype="int64")) # encoder_ids
export_path = os.path.join(training_args.output_dir, 'export')
trainer.export_model(export_path,
input_spec=input_spec,
diff --git a/applications/text_classification/multi_label/few-shot/utils.py b/applications/text_classification/multi_label/few-shot/utils.py
index 6891c22829fc..4855e43e7bf3 100644
--- a/applications/text_classification/multi_label/few-shot/utils.py
+++ b/applications/text_classification/multi_label/few-shot/utils.py
@@ -15,7 +15,6 @@
import os
from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
def load_local_dataset(data_path, splits, label_list):
@@ -39,14 +38,14 @@ def _reader(data_file, label_list):
for idx, line in enumerate(fp):
data = line.strip().split("\t")
if len(data) == 1:
- yield InputExample(text_a=data[0])
+ yield {"text_a": data[0]}
else:
text, label = data
label = label.strip().split(",")
label = [
float(1) if x in label else float(0) for x in label_list
]
- yield InputExample(text_a=text, labels=label)
+ yield {"text_a": text, "labels": label}
split_map = {"train": "train.txt", "dev": "dev.txt", "test": "test.txt"}
datasets = []
diff --git a/docs/advanced_guide/prompt.md b/docs/advanced_guide/prompt.md
index e45aca61d4b6..c8b720261302 100644
--- a/docs/advanced_guide/prompt.md
+++ b/docs/advanced_guide/prompt.md
@@ -22,11 +22,11 @@ Prompt API 提供了这类算法实现的基本模块,支持[PET](https://arxi
* [如何定义模板](#如何定义模板)
* [离散型模板](#离散型模板)
* [连续型模板](#连续型模板)
+ * [前缀连续型模板](#前缀连续型模板)
* [快速定义模板](#快速定义模板)
* [如何定义标签词映射](#如何定义标签词映射)
- * [单掩码映射](#单掩码映射)
- * [多掩码映射](#多掩码映射)
- * [标签词映射分类](#标签词映射分类)
+ * [离散型标签词映射](#离散型标签词映射)
+ * [连续型标签词映射](#连续型标签词映射)
* [快速开始训练](#快速开始训练)
* [数据准备](#数据准备)
* [预训练参数准备](#预训练参数准备)
@@ -39,18 +39,28 @@ Prompt API 提供了这类算法实现的基本模块,支持[PET](https://arxi
## 如何定义模板
-**模板**(Template)的功能是在原有输入文本上增加提示语句,从而将原任务转化为 MLM 任务,可以分为离散型和连续型两种。Prompt API 中提供了统一的数据结构来构造不同类型的模板,输入相应格式的**字符串**,通过解析得到对应的输入模板,即字典构成的列表。
+**模板**(Template)的功能是在原有输入文本上增加提示语句,从而将原任务转化为 MLM 任务,可以分为离散型和连续型两种。Prompt API 中提供了统一的数据结构来构造不同类型的模板,输入相应格式的**字符串**,通过解析得到对应的输入模板。模板由不同字段构成,可任意组合。每个字段中的关键字定义了数据文本或者提示文本,即 `input_ids`,属性可定义该字段是否可截断,以及对应的 `position_ids`,`token_type_ids` 等。
### 离散型模板
离散型模板 `ManualTemplate` 是直接将提示语句与原始输入文本拼接起来,二者的词向量矩阵共享,均为预训练模型学到的词向量矩阵。可用于实现 PET、RGL 等算法。
-**模板关键字**
+**模板关键字及属性**
-- ``text`` :数据集中原始输入文本对应的关键字,包括`text_a`和`text_b`。[数据准备](#数据准备)中介绍了如何将自定义数据集转化为统一格式。
-- ``hard`` :自定义的文本提示语句。
+- ``text`` :数据集中原始输入文本对应的关键字,例如,`text_a`、`text_b` 和 `content`。
+- ``hard`` :自定义的提示语句文本。
- ``mask`` :待预测词的占位符。
-- ``sep`` :用于区分不同的句子。`sep`前后的句子对应不同的`token_type_id`。
+ - ``length`` :定义 ``mask`` 的数量。
+- ``sep`` :句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义,默认相同。
+- ``options`` :数据集字典或者文件中的候选标签序列。
+ - ``add_omask`` :在每个标签前新增 `[O-MASK]` 字符,用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+ - ``add_prompt`` :给每个标签拼接固定的提示文本,标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时,当前字段是否可截断。可选 `True` 和 `False`。
**模板定义**
@@ -64,16 +74,25 @@ Prompt API 提供了这类算法实现的基本模块,支持[PET](https://arxi
“{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}
```
+```
+{'options': './data/label.txt'}{'sep'}下边两句话间的逻辑关系是什么?{'text': 'text_a'}{'sep': None, 'token_type': 1}{'text': 'text_b'}
+```
+其中 `label.txt` 为候选标签的本地文件路径,每行一个候选标签,例如
+
+```
+中立
+蕴含
+矛盾
+```
+
**样本示例**
例如,对于自然语言推理任务,给定样本
```python
-from paddlenlp.prompt import InputExample
-sample = InputExample(uid=0,
- text_a="心里有些生畏,又不知畏惧什么",
- text_b="心里特别开心",
- labels="矛盾")
+sample = {
+ "text_a": "心里有些生畏,又不知畏惧什么", "text_b": "心里特别开心", "labels": "矛盾"
+}
```
按照模板修改拼接后,最终输入模型的文本数据为
@@ -90,17 +109,17 @@ from paddlenlp.prompt import ManualTemplate
from paddlenlp.transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-template = ManualTemplate(tokenizer=tokenizer,
- max_seq_length=512,
- template="“{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}")
-input_dict = template.wrap_one_example(sample)
+template = ManualTemplate(prompt="“{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}",
+ tokenizer=tokenizer,
+ max_length=512)
+input_dict = template(sample)
```
其中初始化参数定义如下
+- ``prompt`` :定义提示语句以及与输入文本组合方式的字符串。
- ``tokenizer`` :预训练模型的 tokenizer,用于文本编码。
-- ``max_seq_length`` :定义输入模型文本的最大长度,包括提示部分。当输入长度超过最大长度时,只会截断`text`关键字对应的输入文本,提示部分不做处理。
-- ``template`` :定义提示语句以及与输入文本组合方式的字符串。
+- ``max_length`` :定义输入模型文本的最大长度,包括提示部分。
**使用技巧**
@@ -115,24 +134,42 @@ input_dict = template.wrap_one_example(sample)
**模板关键字**
-- ``text`` :数据集中原始输入文本对应的关键字,包括`text_a`和`text_b`。[数据准备](#数据准备)中介绍了如何将自定义数据集转化为统一格式。
+- ``text`` :数据集中原始输入文本对应的关键字,例如,`text_a`和`text_b`。
- ``hard`` :自定义的文本提示语句。
- ``mask`` :待预测词的占位符。
-- ``sep`` :用于区分不同的句子。`sep`前后的句子对应不同的`token_type_id`。
-- ``soft`` 表示连续型提示。若值为 ``None`` ,则使用对应数量的随机初始化向量作为提示;若值为文本,则使用对应长度的连续性向量作为提示,并预训练词向量中文本对应的向量进行初始化。
+- ``sep`` :句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义,默认相同。
+- ``soft`` 表示连续型提示。若值为 ``None`` ,则随机初始化提示向量;若值为文本,则使用文本对应的预训练字向量初始化提示向量。
+ - ``length`` :定义 ``soft token`` 的数量。若定义文本长度小于该值,超过部分随机初始化。
+ - ``encoder`` :定义 `soft token` 的编码器类型,可选 `lstm`,`mlp`。默认为 `None`, 不使用编码器。
+ - ``hidden_size`` :定义编码器的隐藏层维度。默认与预训练词向量维度相同。
+- ``options`` :数据集字典或者文件中的候选标签序列。
+ - ``add_omask`` :在每个标签前新增 `[O-MASK]` 字符,用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+ - ``add_prompt`` :给每个标签拼接固定的提示文本,标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时,当前字段是否可截断。可选 `True` 和 `False`。
**模板定义**
- 定义长度为 1 的连续型提示,随机初始化:
```python
-"{'soft': None}{'text': 'text_a'}{'sep'}{'text': 'text_b'}"
+"{'soft'}{'text': 'text_a'}{'sep': None, 'token_type': 1}{'text': 'text_b'}"
+```
+
+- 定义长度为 10 的连续型提示,随机初始化,编码器为 `mlp`:
+
+```python
+"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': None, 'length':10, 'encoder': 'mlp'}{'mask'}"
```
-- 定义长度为 10 的连续型提示,随机初始化,其中 ``duplicate`` 参数表示连续型提示的长度(仅在随机初始化时有效,即`soft`值为`None`):
+- 定义长度为 15 的连续型提示,使用 `请判断` 初始化前三个 soft token,其余随机初始化,编码器为隐藏层维度为 100 的双层 LSTM:
```python
-"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': None, `duplicate`:10}{'mask'}"
+"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断:', 'length': 15, 'encoder': 'lstm', 'hidden_size': 100}{'mask'}"
```
- 定义长度为 15 的连续型提示,使用 `"请判断这两个句子间的逻辑关系:"` 的预训练词向量逐一进行初始化:
@@ -156,38 +193,98 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-template = SoftTemplate(tokenizer=tokenizer,
- max_seq_length=512,
- model=model,
- template="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系:'}{'mask'}",
- prompt_encoder='lstm',
- encoder_hidden_size=200)
+template = SoftTemplate(prompt="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系:'}{'mask'}",
+ tokenizer=tokenizer,
+ max_length=512,
+ word_embeddings=model.get_input_embeddings())
```
其中初始化参数定义如下
+- ``prompt`` :定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
- ``tokenizer`` :预训练模型的 tokenizer,用于文本编码。
-- ``max_seq_length`` :定义输入模型文本的最大长度,包括提示部分。当输入长度超过最大长度时,只会截断`text`关键字对应的输入文本,提示部分不做处理。
-- ``model`` : 预训练语言模型,为了取预训练词向量用于连续型提示向量初始化。
-- ``template`` :定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
-- ``prompt_encoder`` : 连续型提示向量的编码器,可选 ``mlp`` 和 ``lstm``。默认为 ``None`` ,即无编码器,直接使用向量。
-- ``encoder_hidden_size`` : 连续型提示向量的维度。默认为 ``None`` ,即与预训练词向量维度相同。
+- ``max_seq_length`` :定义输入模型文本的最大长度,包括提示部分。
+- ``word_embeddings`` :预训练语言模型的词向量,用于连续型提示向量初始化。
+- ``soft_embeddings`` :连续型提示向量矩阵,可用于不同模板间的连续型参数共享。设置后将覆盖默认连续型向量矩阵。
**使用技巧**
- 对于分类任务,推荐的连续型提示长度一般为10-20。
- 对于随机初始化的连续性 prompt 向量,通常用比预训练模型微调更大的学习率来更新参数。
- 与离散型模板相似,连续型模板对初始化参数也比较敏感。自定义提示语句作为连续性 prompt 向量的初始化参数通常比随机初始化效果好。
-- prompt_encoder 为已有论文中的策略,用于建模不同连续型提示向量之间的序列关系。在实际应用中推荐先去掉 prompt_encoder 调整向量初始化。
+- prompt_encoder 为已有论文中的策略,用于建模不同连续型提示向量之间的序列关系。
+
+
+### 前缀连续型模板
+
+`PrefixTemplate` 同样使用了连续型向量作为提示,与 `SoftTemplate` 的不同,该模版的提示向量不仅仅作用于输入层,每层都会有相应的提示向量。可用于实现 P-Tuning 等算法。
+
+**模板关键字**
+
+- ``text`` :数据集中原始输入文本对应的关键字,例如,`text_a`和`text_b`。
+- ``hard`` :自定义的文本提示语句。
+- ``mask`` :待预测词的占位符。
+- ``sep`` :句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义,默认相同。
+- ``prefix`` 表示连续型提示,该字段**必须**位于模板首位。若值为 ``None`` ,则随机初始化提示向量;若值为文本,则使用文本对应的预训练字向量初始化提示向量。
+ - ``length`` :定义 ``soft token`` 的数量。若定义文本长度小于该值,超过部分随机初始化。
+ - ``encoder`` :定义 `soft token` 的编码器类型,可选 `lstm`,`mlp`。默认为 `None`, 不使用编码器。
+ - ``hidden_size`` :定义编码器的隐藏层维度。默认与预训练词向量维度相同。
+- ``options`` :数据集字典或者文件中的候选标签序列。
+ - ``add_omask`` :在每个标签前新增 `[O-MASK]` 字符,用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+ - ``add_prompt`` :给每个标签拼接固定的提示文本,标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时,当前字段是否可截断。可选 `True` 和 `False`。
+
+**模板定义**
+
+- 定义长度为 15 的连续型提示,随机初始化:
+
+```python
+"{'prefix': '新闻类别', 'length': 10, 'encoder': 'lstm'}{'text': 'text_a'}"
+```
+
+- 定义混合模板,这里`prefix`关键字对应的提示和`hard`对应的提示对应两套不同的向量:
+
+```python
+"{'prefix': '自然语言推理任务:', 'encoder': 'mlp'}{'text': 'text_a'}{'sep'}{'text': 'text_b'}这两个句子间的逻辑关系是{'mask'}"
+```
+
+
+**调用 API**
+
+```python
+from paddlenlp.prompt import PrefixTemplate
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
+
+model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
+tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
+template = PrefixTemplate(prompt="{'prefix': '任务描述'}{'text': 'text_a'}{'mask'}",
+ tokenizer=tokenizer,
+ max_length=512,
+ model=model,
+ prefix_dropout=0.1)
+```
+
+其中初始化参数定义如下
+
+- ``prompt`` :定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
+- ``tokenizer`` :预训练模型的 tokenizer,用于文本编码。
+- ``max_length`` :定义输入模型文本的最大长度,包括提示部分。
+- ``model`` :预训练语言模型,用于连续型提示向量初始化,以及根据模型结构生成每层对应的提示向量。
+- ``prefix_dropout`` :连续型提示向量的丢弃概率,用于正则化。
### 快速定义模板
-PaddleNLP 提供了 ``AutoTemplate`` API 以便快速定义单句输入的手工初始化的连续型模板,同时支持直接按照模板类型自动切换离散型模板和离散型模板。
+PaddleNLP 提供了 ``AutoTemplate`` API 快速定义简化离散型模板,也可根据完整模板字符串自动切换 ManualTemplate、SoftTemplate 和 PrefixTemplate。
**模板定义**
-- 只定义用于初始化连续型向量的文本提示,即可得到拼接到句尾的连续型模板输入。例如,
+- 快速定义离散型的文本提示。例如,
```python
"这篇文章表达了怎样的情感?"
@@ -196,7 +293,7 @@ PaddleNLP 提供了 ``AutoTemplate`` API 以便快速定义单句输入的手工
等价于
```python
-"{'text': 'text_a'}{'soft': '这篇文章表达了怎样的情感?'}{'mask'}"
+"{'text': 'text_a'}{'hard': '这篇文章表达了怎样的情感?'}{'mask'}"
```
- 当输入为完整模板字符串时,解析得到的模板与[离散型模板](#离散型模板)和[连续型模板](#连续型模板)中描述的一致。
@@ -210,40 +307,37 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
# 离散型模板,返回值为 ManualTemplate 实例
-template = AutoTemplate.create_from(template="{'text': 'text_a'}和{'text': 'text_b'}之间的逻辑关系是{'mask'}",
+template = AutoTemplate.create_from(prompt="这个句子表达了怎样的情感?",
+ tokenizer=tokenizer,
+ max_length=512)
+
+template = AutoTemplate.create_from(prompt="这个句子表达了怎样的情感?{'text': 'text_a'}{'mask'}",
tokenizer=tokenizer,
- max_seq_length=512)
+ max_length=512)
# 连续型模板,返回值为 SoftTemplate 实例
-template = AutoTemplate.create_from(template="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系:'}{'mask'}",
+template = AutoTemplate.create_from(prompt="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系:'}{'mask'}",
tokenizer=tokenizer,
- max_seq_length=512,
- model=model,
- prompt_encoder='lstm',
- encoder_hidden_size=200)
+ max_length=512,
+ model=model)
-# 快速定义单句连续型模板,返回值为 SoftTemplate 实例
-template = AutoTemplate.create_from(template="这篇文章表达了怎样的情感?",
+# 前缀连续型模板,返回值为 PrefixTemplate 实例
+template = AutoTemplate.create_from(prompt="{'prefix': None, 'encoder': 'mlp', 'hidden_size': 50}{'text': 'text_a'}",
tokenizer=tokenizer,
- max_seq_length=512,
- model=model,
- prompt_encoder='lstm',
- encoder_hidden_size=200)
+ max_length=512,
+ model=model)
```
其中初始化参数定义如下
+- ``prompt`` :定义离散型/连续型提示、初始化以及和输入文本的组合方式。
- ``tokenizer`` :预训练模型的 tokenizer,用于文本编码。
-- ``max_seq_length`` :定义输入模型文本的最大长度,包括提示部分。当输入长度超过最大长度时,只会截断`text`关键字对应的输入文本,提示部分不做处理。
+- ``max_length`` :定义输入模型文本的最大长度,包括提示部分。
- ``model`` :预训练语言模型,为了取预训练词向量用于连续型提示向量初始化。
-- ``template`` :定义离散型/连续型提示、初始化以及和输入文本的组合方式。
-- ``prompt_encoder`` :连续型提示向量的编码器,可选 ``mlp`` 和 ``lstm`` 。默认为 ``None`` ,即无编码器,直接使用向量。
-- ``encoder_hidden_size`` :连续型提示向量的维度。默认为 ``None`` ,即与预训练词向量维度相同。
-
## 如何定义标签词映射
-**标签词映射**(Verbalizer)也是提示学习中可选的重要模块,用于建立预测词和标签之间的映射,将“预训练-微调”模式中预测标签的任务转换为预测模板中掩码位置的词语,从而将下游任务统一为预训练任务的形式。目前框架支持了离散型标签词映射和 [Word-level Adversarial ReProgramming (WARP)](https://aclanthology.org/2021.acl-long.381/) 方法。
+**标签词映射**(Verbalizer)也是提示学习中可选的重要模块,用于建立预测词和标签之间的映射,将“预训练-微调”模式中预测标签的任务转换为预测模板中掩码位置的词语,从而将下游任务统一为预训练任务的形式。目前框架支持了离散型标签词映射和连续型标签词映射 [Word-level Adversarial ReProgramming (WARP)](https://aclanthology.org/2021.acl-long.381/) 方法。
例如,在情感二分类任务中,微调方法和提示学习的标签体系如下
@@ -259,9 +353,9 @@ template = AutoTemplate.create_from(template="这篇文章表达了怎样的情
具体来说,对于模板 ``{'text':'text_a'}这句话表示我{'mask'}满意。`` ,我们使用映射 ``{'负向': '不', '正向': '很'}`` 将标签 ``负向`` 映射为 ``不`` ,将标签 ``正向`` 映射为 ``很`` 。也就是说,我们期望对于正向情感的文本,预测结果为 ``...这句话表示我很满意。`` ,对于负向情感的文本,预测结果为 ``...这句话表示我不满意。``
-### 单掩码映射
+### 离散型标签词映射
-``ManualVerbalizer`` 支持构造简单的单 ``{'mask'}`` 标签词映射,直接作用于 ``AutoMaskedLM`` 模型结构。当标签对应的预测词长度大于 ``1`` 时取均值。
+``ManualVerbalizer`` 支持构造 ``{'mask'}`` 对应的标签词映射,支持多``{'mask'}``,直接作用于 ``AutoMaskedLM`` 模型结构。当标签对应的预测词长度大于 ``1`` 时,默认取均值。
**调用 API**
@@ -271,41 +365,15 @@ from paddlenlp.transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
verbalizer = ManualVerbalizer(tokenizer=tokenizer,
- labels=['负向', '正向'],
- label_words={'负向': '不', '正向': '很'},
- prefix=None)
+ label_words={'负向': '不', '正向': '很'})
```
其中初始化参数定义如下
+- ``label_words`` : 原标签到预测词之间的映射字典。
- ``tokenizer`` : 预训练模型的 tokenizer,用于预测词的编码。
-- ``labels`` : 数据集的原标签列表(可选)。
-- ``label_words`` : 原标签到预测词之间的映射字典。如果同时定义了 ``labels`` ,二者的标签集合需要相同。
-- ``prefix`` : 预测词解码前增加的前缀,用于 ``RoBERTa`` 等对前缀敏感的模型,例如 `roberta-large`, `good` 和 ` good` 经过 tokenize 会得到不同的 id。默认为 ``None`` ,无前缀。
-
-### 多掩码映射
-
-``MultiMaskVerbalizer`` 继承自 ``ManualVerbalizer`` ,支持多 ``{'mask'}`` 标签词映射。预测词长度需与 ``{'mask'}`` 长度一致。
-
-**调用 API**
-
-```python
-from paddlenlp.prompt import MultiMaskVerbalizer
-from paddlenlp.transformers import AutoTokenizer
-
-tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-verbalizer = MultiMaskVerbalizer(tokenizer=tokenizer,
- labels=['负向', '正向'],
- label_words={'负向': '生气', '正向': '高兴'},
- prefix=None)
-```
-
-
-其中初始化参数定义同[单掩码映射](#单掩码映射) 。
-
-
-### 标签词映射分类
+### 连续型标签词映射
标签词映射分类器 ``SoftVerbalizer`` 修改了原 ``AutoMaskedLM`` 的模型结构,将预训练模型最后一层“隐藏层-词表”替换为“隐藏层-标签”的映射。该层网络的初始化参数由标签词映射中的预测词词向量来决定,如果预测词长度大于 ``1`` ,则使用词向量均值进行初始化。当前支持的预训练模型包括 ``ErnieForMaskedLM`` 、 ``BertForMaskedLM`` 、 ``AlbertForMaskedLM`` 和 ``RobertaForMaskedLM`` 。可用于实现 WARP 算法。
@@ -318,15 +386,13 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-verbalizer = SoftVerbalizer(tokenizer=tokenizer,
- model=model,
- labels=['负向', '正向'],
- label_words={'负向': '生气', '正向': '高兴'},
- prefix=None)
+verbalizer = SoftVerbalizer(label_words={'负向': '生气', '正向': '高兴'},
+ tokenizer=tokenizer,
+ model=model)
```
-其中初始化参数定义同[单掩码映射](#单掩码映射) ,此外
-
+- ``label_words`` : 原标签到预测词之间的映射字典。
+- ``tokenizer`` : 预训练模型的 tokenizer,用于预测词的编码。
- ``model`` :预训练语言模型,用于取预训练词向量进行“隐藏层-标签”网络的修改和初始化。
## 快速开始训练
@@ -335,29 +401,24 @@ verbalizer = SoftVerbalizer(tokenizer=tokenizer,
### 数据准备
-Prompt 框架定义了统一的样本结构 ``InputExample`` 以便进行数据处理,数据集样本需要封装在 ``MapDataset`` 中。
+数据集封装为 ``MapDataset`` 类型。每条数据格式为字典结构,字典中关键字与模板中 `text` 定义的值相对应,统一使用 `labels` 关键字表示样本标签。
-例如,对于文本语义相似度 BUSTM 数据集中的原始样本
+例如,文本语义相似度 BUSTM 数据集中的数据样本
```python
-data = [
+from paddlenlp.datasets import MapDataset
+
+data_ds = MapDataset([
{'id': 3, 'sentence1': '你晚上吃了什么', 'sentence2': '你晚上吃啥了', 'label': 1},
{'id': 4, 'sentence1': '我想打开滴滴叫的士', 'sentence2': '你叫小欧吗', 'label': 0},
{'id': 5, 'sentence1': '女孩子到底是不是你', 'sentence2': '你不是女孩子吗', 'label': 1}
-]
-```
+])
+def convert_label_keyword(input_dict):
+ input_dict["labels"] = input_dict.pop("label")
+ return input_dict
-需要转换为统一格式
-
-```python
-from paddlenlp.datasets import MapDataset
-from paddlenlp.prompt import InputExample
-
-data_ds = MapDataset([InputExample(uid=example["id"],
- text_a=example["sentence1"],
- text_b=example["sentence2"],
- labels=example["label"]) for example in data])
+data_ds = data_ds.map(convert_label_keyword)
```
### 预训练参数准备
@@ -383,13 +444,13 @@ from paddlenlp.prompt import ManualVerbalizer
from paddlenlp.prompt import PromptModelForSequenceClassification
# 定义模板
-template = AutoTemplate.create_from(template="{'text': 'text_a'}和{'text': 'text_b'}说的是{'mask'}同的事情。",
+template = AutoTemplate.create_from(prompt="{'text': 'text_a'}和{'text': 'text_b'}说的是{'mask'}同的事情。",
tokenizer=tokenizer,
- max_seq_length=512)
+ max_length=512)
# 定义标签词映射
-verbalizer = ManualVerbalizer(tokenizer=tokenizer,
- label_words={0: '不', 1: '相'})
+verbalizer = ManualVerbalizer(label_words={0: '不', 1: '相'},
+ tokenizer=tokenizer)
# 定义文本分类提示模型
prompt_model = PromptModelForSequenceClassification(model,
@@ -404,8 +465,8 @@ prompt_model = PromptModelForSequenceClassification(model,
- ``model`` : 预训练模型实例,支持 ``AutoModelForMaskedLM`` 和 ``AutoModelForSequenceClassification`` 。
- ``template`` : 模板实例。
- ``verbalizer`` : 标签词映射实例。当设为 ``None`` 时,不使用标签词映射,模型输出及损失值计算由 ``model`` 类型定义。
-- ``freeze_plm`` : 在训练时是否固定预训练模型参数。对于规模较小的预训练模型,推荐更新预训练模型参数。
-- ``freeze_dropout`` : 在训练时是否固定预训练模型参数并关闭 ``dropout`` 。 当 ``freeze_dropout=True`` ,``freeze_plm`` 也为 ``True`` 。
+- ``freeze_plm`` : 在训练时固定预训练模型参数,默认为 `False`。对于轻量级预训练模型,推荐使用默认值。
+- ``freeze_dropout`` : 在训练时固定预训练模型参数并关闭 ``dropout`` 。 当 ``freeze_dropout=True`` ,``freeze_plm`` 也为 ``True`` 。
### 使用PromptTrainer训练
@@ -497,6 +558,8 @@ if training_args.do_train:
- WARP: Word-level Adversarial ReProgramming. [[PDF]](https://aclanthology.org/2021.acl-long.381/)
- RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning. [[PDF]](https://aclanthology.org/2022.findings-naacl.81/)
- R-Drop: Regularized Dropout for Neural Networks. [[PDF]](https://arxiv.org/abs/2106.14448)
+- Openprompt: An open-source framework for prompt-learning. [[PDF]](https://arxiv.org/abs/2111.01998)
+
### 附录
@@ -506,9 +569,9 @@ if training_args.do_train:
| 参数 | 类型 | 默认值 | 含义 |
| ---------------- | ------ | ------- | ------------------------------------------------------- |
-| max_seq_length | int | 512 | 模型输入的最大长度,包括模板部分 |
-| freeze_plm | bool | False | 是否在训练时固定预训练模型的参数 |
-| freeze_dropout | bool | False | 是否在训练时固定预训练模型的参数,同时关闭 dropout |
+| max_seq_length | int | 512 | 模型输入的最大长度,包括模板部分 |
+| freeze_plm | bool | False | 是否在训练时固定预训练模型的参数 |
+| freeze_dropout | bool | False | 是否在训练时固定预训练模型的参数,同时关闭 dropout |
| use_rdrop | bool | False | 是否使用 RDrop 策略,详见 [RDrop 论文](https://arxiv.org/abs/2106.14448) |
| alpha_rdrop | float | 5.0 | RDrop Loss 的权重 |
| use_rgl | bool | False | 是否使用 RGL 策略,详见 [RGL 论文](https://aclanthology.org/2022.findings-naacl.81/) |
diff --git a/docs/trainer.md b/docs/trainer.md
index f141d15a665d..96566548f811 100644
--- a/docs/trainer.md
+++ b/docs/trainer.md
@@ -395,6 +395,37 @@ Trainer 是一个简单,但功能完整的 Paddle训练和评估模块,并
The value of initial scale_loss for fp16. (default: 32768)
+ --sharding
+ 是否使用Paddle的Sharding数据并行功能,用户的参数。支持sharding `stage1`, `stage2` or `stage3`。
+ 其中`stage2``stage3`可以和`offload`组合使用。
+ 每个种策略分别为:
+ stage1 : optimizer 中的参数切分到不同卡
+ stage2 : optimizer + gradient 中的参数切分到不同卡
+ stage3 : parameter + gradient + optimizer 中的参数都切分到不同卡
+ offload : offload parameters to cpu 部分参数存放到cpu中
+ (`str`, 可选, 默认为 `` 不使用sharding)
+ 注意:当前stage3暂时不可用
+
+ Whether or not to use Paddle Sharding Data Parallel training (in distributed training
+ only). The base option should be `stage1`, `stage2` or `stage3` and you can add
+ CPU-offload to `stage2` or `stage3` like this: `stage2 offload` or `stage3 offload`.
+ Each stage means:
+ stage1 : optimizer state segmentation
+ stage2 : optimizer state + gradient segmentation
+ stage3 : parameter + gradient + optimizer state segmentation
+ offload : offload parameters to cpu
+ NOTICE: stage3 is temporarily unavaliable.
+
+ --sharding_degree
+ 设置sharding的通信组参数,表示通信组的大小。同一个sharding通信组内的参数,进行sharding,分布到不同卡上。
+ 不同sharding通信组之间,相当于单纯的数据并行。此选项只在sharding选项开启时候生效。
+ 默认值为-1,表示所有训练的卡在同一个通信组内。
+ (`int`, 可选, 默认为 `-1`)
+
+ Sharding parameter in certain cards group. For example, aussume we use 2 machines each
+ with 8 cards, then set sharding_degree=8, sharding will only communication inside machine.
+ default -1 means sharding parameters between all workers. (`int`, *optional*, defaults to `-1`)
+
--recompute
是否使用重计算训练。可以节省显存。
重新计算前向过程以获取梯度,减少中间变量显存
diff --git a/examples/language_model/moe/dygraph/framework/group_sharded.py b/examples/language_model/moe/dygraph/framework/group_sharded.py
index e56e284d7942..682a70b53efc 100644
--- a/examples/language_model/moe/dygraph/framework/group_sharded.py
+++ b/examples/language_model/moe/dygraph/framework/group_sharded.py
@@ -35,7 +35,6 @@
from paddle.fluid.clip import ClipGradBase, _squared_l2_norm
from paddle.fluid.dygraph import base as imperative_base
from paddle.fluid import core, layers, framework
-from paddle.distributed import collective
from paddle.incubate.distributed.models.moe.grad_clip import ClipGradForMOEByGlobalNorm
# Old version
@@ -72,18 +71,19 @@ def _dygraph_clip(self, params_grads):
global_norm_var_normal, sum_dtype \
= self.get_l2_norm_pow(normal_params_grads)
if global_norm_var_normal is not None:
- collective.all_reduce(global_norm_var_normal,
- op=collective.ReduceOp.SUM,
- group=self.moe_group)
+ paddle.distributed.all_reduce(global_norm_var_normal,
+ op=paddle.distributed.ReduceOp.SUM,
+ group=self.moe_group)
global_norm_var_moe = None
if len(moe_params_grads) > 0:
global_norm_var_moe, _ \
= self.get_l2_norm_pow(moe_params_grads, sum_dtype)
if global_norm_var_moe is not None:
- collective.all_reduce(global_norm_var_moe,
- op=collective.ReduceOp.SUM,
- group=self.moe_group)
+ paddle.distributed.all_reduce(
+ global_norm_var_moe,
+ op=paddle.distributed.ReduceOp.SUM,
+ group=self.moe_group)
if global_norm_var_normal is None and global_norm_var_moe is None:
return params_grads
diff --git a/examples/language_model/t5/README.md b/examples/language_model/t5/README.md
index 313a47c0079d..ef4ca649585f 100644
--- a/examples/language_model/t5/README.md
+++ b/examples/language_model/t5/README.md
@@ -46,12 +46,53 @@ python run_glue.py \
- `scheduler_type` scheduler类型,可选linear和cosine,默认linear。
- `output_dir` 表示模型保存路径。
+使用trainer进行Fine-tuning:
+```shell
+python -m paddle.distributed.launch --gpus "0,1,2,3" run_glue_trainer.py \
+ --model_name_or_path t5-base \
+ --task_name rte \
+ --max_seq_length 256 \
+ --do_train \
+ --do_eval \
+ --per_device_train_batch_size 16 \
+ --per_device_eval_batch_size 64 \
+ --learning_rate 1e-4 \
+ --weight_decay 0.01 \
+ --warmup_ratio 0.1 \
+ --num_train_epochs 10 \
+ --eval_steps 200 \
+ --logging_steps 20 \
+ --save_steps 200 \
+ --save_total_limit 3 \
+ --metric_for_best_model "eval_accuarcy" \
+ --fp16 false \
+ --fp16_opt_level "O1" \
+ --recompute true \
+ --sharding "stage1" \
+ --overwrite_output_dir \
+ --disable_tqdm true \
+ --output_dir outputs/rte/
+```
+具体参数含义请参见: https://paddlenlp.readthedocs.io/zh/latest/trainer.html
+
###### t5-base模型在GLUE开发集上的结果:
| Model | cola | sst-2 | mrpc | sts-b | qqp | mnli | qnli | rte | mean |
|--------------------------------|-------|-------|-------------|------------------|-------------|-------------|------|-------|-------|
| | mcc | acc | acc | pearson | acc | acc | acc | acc | |
| T5-base-Paddle | 61.74 | 95.18 | 90.44 | 90.09 | 91.60 | 87.18 | 93.56 | 81.95 | 86.4675 |
+###### t5_v1_1-base模型在GLUE开发集上的结果:
+使用`run_glue_trainer.py`运行,由于`t5_v1_1-base`没有在glue任务上进行训练过,直接生成label的策略需要的训练时间需要更长。
+| Model | cola | sst-2 | mrpc | sts-b | qqp | mnli | qnli | rte |
+|--------------------------------|-------|-------|-------------|------------------|-------------|-------------|------|-------|
+| | mcc | acc | acc | pearson | acc | acc | acc | acc |
+| T5-v1_1-base Paddle | 47.6845 | 94.38 | 84.31 | 87.74 | 88.05 | 85.39 | 90.518 | 65.70 |
+| epoch | 100 | 10 | 100 | 100 | 3 | 3 | 10 | 100 |
+
+注:
+- 直接生成label的finetune方式难度较大,前期基本学习如何正确生成label标签,后期才学习分类任务。
+- 生成的label标签设计,标签差异大一些,效果会更好一些。
+- `qqp`,`mnli`数据集适当增大训练epoch数,可以取得更好效果。
### GLUE Demo测试
diff --git a/examples/language_model/t5/data.py b/examples/language_model/t5/data.py
index 2927533f7626..9df3329d80de 100644
--- a/examples/language_model/t5/data.py
+++ b/examples/language_model/t5/data.py
@@ -50,6 +50,33 @@
),
])
+GLUE_1_1_PROCESSED = collections.OrderedDict([
+ ("cola", (["cola sentence: "], ["outrageous", "acceptable"])),
+ ("sst-2", (["sst2 sentence: "], ["negative", "positive"])),
+ (
+ "mrpc",
+ (["mrpc sentence1: ", " sentence2: "], ["nonidentical", "equivalent"]),
+ ),
+ ("sts-b", (["stsb sentence1: ", " sentence2: "], None)),
+ ("qqp", (["qqp question1: ", " question2: "], ["inequable", "duplicate"])),
+ (
+ "mnli",
+ (
+ ["mnli hypothesis: ", " premise: "],
+ ["contradiction", "entailment", "neutral"],
+ ),
+ ),
+ (
+ "qnli",
+ (["qnli question: ", " sentence: "], ["entailment", "contradiction"]),
+ ),
+ (
+ "rte",
+ (["rte sentence1: ",
+ " rte sentence2: "], ["entailment", "contradiction"]),
+ ),
+])
+
def trans_func(example, tokenizer, args):
task_name = args.task_name
diff --git a/examples/language_model/t5/run_glue_trainer.py b/examples/language_model/t5/run_glue_trainer.py
new file mode 100644
index 000000000000..712062ff7320
--- /dev/null
+++ b/examples/language_model/t5/run_glue_trainer.py
@@ -0,0 +1,429 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import math
+import os
+from functools import partial
+from dataclasses import dataclass, field
+from typing import Optional, Dict, Union, Any, List, Tuple
+
+from tqdm import tqdm
+import paddle
+import paddle.nn as nn
+from paddlenlp.transformers import T5ForConditionalGeneration, T5Tokenizer
+from paddlenlp.datasets import load_dataset
+from paddlenlp.trainer import get_last_checkpoint
+from paddlenlp.data import DataCollatorWithPadding, Pad
+from paddlenlp.trainer import (
+ PdArgumentParser,
+ TrainingArguments,
+ Trainer,
+)
+from paddlenlp.utils.log import logger
+
+from utils import load_pickle, save_pickle, GLUE_METRICS
+from data import GLUE_PROCESSED, GLUE_1_1_PROCESSED
+
+label_length_map = {
+ "cola": 4,
+ "sst-2": 1,
+ "mrpc": 5,
+ "sts-b": 5,
+ "qqp": 5,
+ "mnli": 4,
+ "qnli": 5,
+ "rte": 5,
+}
+
+
+def trans_func(example, tokenizer, args):
+ task_name = args.task_name
+ PROCESSED = GLUE_PROCESSED
+ if "v1_1" in args.cache_dir:
+ PROCESSED = GLUE_1_1_PROCESSED
+ processed, label = PROCESSED[task_name]
+ if label:
+ id2label = dict(zip(range(len(label)), label))
+ else:
+ id2label = None
+
+ is_test = "labels" not in example
+
+ if not is_test:
+ if id2label:
+ label_text = id2label[example["labels"]]
+ else:
+ label_text = str(example["labels"])
+ target = tokenizer(label_text,
+ return_token_type_ids=False,
+ return_attention_mask=True)
+
+ if len(processed) == 1:
+ text = processed[0] + example["sentence"]
+ else:
+ text = processed[0] + example["sentence1"] + processed[1] + example[
+ "sentence2"]
+
+ source = tokenizer(
+ text,
+ max_seq_len=args.max_seq_length,
+ padding='max_length',
+ return_token_type_ids=False,
+ return_attention_mask=True,
+ )
+
+ if not is_test:
+ return {
+ "input_ids": source["input_ids"],
+ "attention_mask": source["attention_mask"],
+ "labels": target["input_ids"],
+ "decoder_attention_mask": target["attention_mask"],
+ }
+ else:
+ return {
+ "input_ids": source["input_ids"],
+ "attention_mask": source["attention_mask"]
+ }
+
+
+class BatchDict(object):
+
+ def __init__(self, fn):
+ assert isinstance(fn, (dict)), 'Input pattern not understood. The input of Dict must be a dict with key of input column name and value of collate_fn ' \
+ 'Received fn=%s' % (str(fn))
+
+ self._fn = fn
+
+ for col_name, ele_fn in self._fn.items():
+ assert callable(
+ ele_fn
+ ), 'Batchify functions must be callable! type(fn[%d]) = %s' % (
+ col_name, str(type(ele_fn)))
+
+ def __call__(self, data):
+
+ ret = {}
+ if len(data) <= 0:
+ return ret
+
+ for col_name, ele_fn in self._fn.items():
+ # skip unused col_name, such as labels in test mode.
+ if col_name not in data[0].keys():
+ continue
+ result = ele_fn([ele[col_name] for ele in data])
+ ret[col_name] = result
+
+ return ret
+
+
+def get_train_dataset(tokenizer, args):
+ filename = os.path.join(args.cache_dir, args.task_name + "_train" + ".pkl")
+
+ if os.path.exists(filename):
+ ds = load_pickle(filename)
+ else:
+ ds = load_dataset("glue", args.task_name, splits="train")
+ ds.map(
+ partial(trans_func, tokenizer=tokenizer, args=args),
+ batched=False,
+ lazy=False,
+ )
+ save_pickle(ds, filename)
+
+ return ds
+
+
+def get_dev_dataset(tokenizer, args):
+ filename = os.path.join(args.cache_dir, args.task_name + "_dev" + ".pkl")
+
+ if os.path.exists(filename):
+ ds = load_pickle(filename)
+ else:
+ ds = load_dataset("glue", args.task_name, splits="dev")
+ ds.map(
+ partial(trans_func, tokenizer=tokenizer, args=args),
+ batched=False,
+ lazy=False,
+ )
+ save_pickle(ds, filename)
+
+ return ds
+
+
+def get_mnli_dev_dataset(tokenizer, args, matched=True):
+ if matched:
+ split = "dev_matched"
+ else:
+ split = "dev_mismatched"
+ filename = os.path.join(args.cache_dir,
+ args.task_name + f"_{split}" + ".pkl")
+ if os.path.exists(filename):
+ ds = load_pickle(filename)
+ else:
+ ds = load_dataset("glue", args.task_name, splits=split)
+ ds.map(
+ partial(trans_func, tokenizer=tokenizer, args=args),
+ batched=False,
+ lazy=False,
+ )
+ save_pickle(ds, filename)
+
+ return ds
+
+
+@dataclass
+class DataArguments:
+ """
+ Arguments pertaining to what data we are going to input our model for training and eval.
+ Using `PdArgumentParser` we can turn this class
+ into argparse arguments to be able to specify them on
+ the command line.
+ """
+
+ task_name: str = field(
+ default=None,
+ metadata={
+ "help": "The name of the task to use (via the datasets library)."
+ })
+
+ max_seq_length: int = field(
+ default=128,
+ metadata={
+ "help":
+ "The maximum total input sequence length after tokenization. Sequences longer "
+ "than this will be truncated, sequences shorter will be padded."
+ },
+ )
+ cache_dir: str = field(default="./caches",
+ metadata={"help": "cache dir for datasets."})
+
+
+@dataclass
+class ModelArguments:
+ """
+ Arguments pertaining to which model/config/tokenizer we are going to fine-tune from.
+ """
+
+ model_name_or_path: str = field(
+ default="t5-small",
+ metadata={
+ "help":
+ "Path to pretrained model or model identifier from https://paddlenlp.readthedocs.io/zh/latest/model_zoo/transformers.html"
+ })
+ export_model_dir: Optional[str] = field(
+ default=None,
+ metadata={
+ "help": "Path to directory to store the exported inference model."
+ },
+ )
+
+
+class T5GlueTrainer(Trainer):
+
+ def __init__(self, do_generation: bool, label2id, **kwargs):
+ super().__init__(**kwargs)
+ self.do_generation = do_generation
+ self.label2id = label2id
+
+ def prediction_step(
+ self,
+ model: nn.Layer,
+ inputs: Dict[str, Union[paddle.Tensor, Any]],
+ prediction_loss_only: bool,
+ ignore_keys: Optional[List[str]] = None,
+ ) -> Tuple[Optional[paddle.Tensor], Optional[paddle.Tensor],
+ Optional[paddle.Tensor]]:
+
+ if not self.do_generation:
+ return super().prediction_step(
+ model,
+ inputs,
+ prediction_loss_only=prediction_loss_only,
+ ignore_keys=ignore_keys)
+
+ all_preds = []
+ all_labels = []
+ # source_ids, source_mask, labels, target_mask = batch
+ labels = inputs["labels"]
+ target_mask = inputs["decoder_attention_mask"]
+
+ with paddle.no_grad():
+ outputs = model.generate(
+ input_ids=inputs["input_ids"],
+ attention_mask=inputs["attention_mask"],
+ max_length=5,
+ )[0]
+
+ for p, l, m in zip(outputs.numpy(), labels.numpy(),
+ target_mask.numpy()):
+ pred = self.tokenizer.decode(p, skip_special_tokens=True).strip()
+ label = self.tokenizer.decode(l[m.astype("bool")],
+ skip_special_tokens=True).strip()
+
+ if self.label2id:
+ # for classifaction task.
+ label = self.label2id[label]
+ if pred not in self.label2id:
+ # set to wrong label if the generated text not in the labal set.
+ pred = 0
+ if label == 0:
+ pred = 1
+ else:
+ pred = self.label2id[pred]
+ else:
+ # for regression task.
+ label = float(label.replace(" ", ""))
+ try:
+ pred = float(pred.replace(" ", ""))
+ except Exception as e:
+ # set to zero if the generated text can not convert to float
+ pred = 0.0
+
+ all_preds.append(pred)
+ all_labels.append(label)
+
+ all_preds = paddle.to_tensor(all_preds).detach()
+ all_labels = paddle.to_tensor(all_labels).detach()
+
+ return (None, all_preds, all_labels)
+
+
+def main():
+ parser = PdArgumentParser(
+ (ModelArguments, DataArguments, TrainingArguments))
+ model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+
+ if "v1_1" in model_args.model_name_or_path:
+ data_args.cache_dir = "./caches_v1_1"
+ if not os.path.exists(data_args.cache_dir):
+ os.mkdir(data_args.cache_dir)
+
+ # Log model and data config
+ training_args.print_config(model_args, "Model")
+ training_args.print_config(data_args, "Data")
+
+ paddle.set_device(training_args.device)
+
+ # Log on each process the small summary:
+ logger.warning(
+ f"Process rank: {training_args.local_rank}, device: {training_args.device}, world_size: {training_args.world_size}, "
+ +
+ f"distributed training: {bool(training_args.local_rank != -1)}, 16-bits training: {training_args.fp16 or training_args.bf16}"
+ )
+
+ # Detecting last checkpoint.
+ last_checkpoint = None
+ if os.path.isdir(
+ training_args.output_dir
+ ) and training_args.do_train and not training_args.overwrite_output_dir:
+ last_checkpoint = get_last_checkpoint(training_args.output_dir)
+ if last_checkpoint is None and len(os.listdir(
+ training_args.output_dir)) > 0:
+ raise ValueError(
+ f"Output directory ({training_args.output_dir}) already exists and is not empty. "
+ "Use --overwrite_output_dir to overcome.")
+ elif last_checkpoint is not None and training_args.resume_from_checkpoint is None:
+ logger.info(
+ f"Checkpoint detected, resuming training at {last_checkpoint}. To avoid this behavior, change "
+ "the `--output_dir` or add `--overwrite_output_dir` to train from scratch."
+ )
+
+ PROCESSED = GLUE_PROCESSED
+ if "v1_1" in data_args.cache_dir:
+ PROCESSED = GLUE_1_1_PROCESSED
+ label_name = PROCESSED[data_args.task_name][1]
+ if label_name:
+ label2id = dict(zip(label_name, range(len(label_name))))
+ else:
+ label2id = None
+ metric_list = GLUE_METRICS[data_args.task_name]
+ generate_max_length = label_length_map[data_args.task_name]
+
+ # get model and tokenizer
+ model = T5ForConditionalGeneration.from_pretrained(
+ model_args.model_name_or_path)
+ tokenizer = T5Tokenizer.from_pretrained(model_args.model_name_or_path)
+
+ # get dataloader
+ train_dataset = get_train_dataset(tokenizer, data_args)
+ if data_args.task_name == "mnli":
+ eval_dataset = get_mnli_dev_dataset(tokenizer, data_args, matched=True)
+ eval_dataset_mismatch = get_mnli_dev_dataset(tokenizer,
+ data_args,
+ matched=False)
+ else:
+ eval_dataset = get_dev_dataset(tokenizer, data_args)
+
+ batchify_fn = lambda samples, fn=BatchDict({
+ "input_ids":
+ Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"), # input_ids
+ "attention_mask":
+ Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+ ), # attention_mask
+ "labels":
+ Pad(axis=0, pad_val=-100, dtype="int64"), # lm_labels
+ "decoder_attention_mask":
+ Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+ ), # decoder_attention_mask
+ }): fn(samples)
+ data_collator = batchify_fn
+
+ # Define the metrics of tasks.
+ def compute_metrics(p):
+ preds = p.predictions[0] if isinstance(p.predictions,
+ tuple) else p.predictions
+
+ results = {}
+ for metric in metric_list:
+ results.update(metric(p.label_ids, preds))
+
+ return results
+
+ trainer = T5GlueTrainer(
+ model=model,
+ criterion=None,
+ args=training_args,
+ data_collator=data_collator,
+ train_dataset=train_dataset if training_args.do_train else None,
+ eval_dataset=eval_dataset if training_args.do_eval else None,
+ tokenizer=tokenizer,
+ compute_metrics=compute_metrics,
+ do_generation=True,
+ label2id=label2id,
+ )
+
+ checkpoint = None
+ if training_args.resume_from_checkpoint is not None:
+ checkpoint = training_args.resume_from_checkpoint
+ elif last_checkpoint is not None:
+ checkpoint = last_checkpoint
+
+ # Training
+ if training_args.do_train:
+ train_result = trainer.train(resume_from_checkpoint=checkpoint)
+ metrics = train_result.metrics
+ # trainer.save_model()
+ trainer.log_metrics("train", metrics)
+ trainer.save_metrics("train", metrics)
+ trainer.save_state()
+
+ # Evaluate and tests model
+ if training_args.do_eval:
+ eval_metrics = trainer.evaluate()
+ trainer.log_metrics("eval", eval_metrics)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/faster_tokenizer/CMakeLists.txt b/fast_tokenizer/CMakeLists.txt
similarity index 86%
rename from faster_tokenizer/CMakeLists.txt
rename to fast_tokenizer/CMakeLists.txt
index c5325955ecca..8a24a4cccd3a 100644
--- a/faster_tokenizer/CMakeLists.txt
+++ b/fast_tokenizer/CMakeLists.txt
@@ -2,9 +2,9 @@ cmake_minimum_required(VERSION 3.10)
project(tokenizers LANGUAGES CXX C VERSION 1.0)
-option(WITH_TESTING "Compile PaddleNLP faster_tokenizer with unit testing" OFF)
-option(WITH_PYTHON "Compile PaddleNLP faster_tokenizer with python interpreter" ON)
-add_definitions(-DFASTERTOKENIZER_LIB)
+option(WITH_TESTING "Compile PaddleNLP fast_tokenizer with unit testing" OFF)
+option(WITH_PYTHON "Compile PaddleNLP fast_tokenizer with python interpreter" ON)
+add_definitions(-DFASTTOKENIZER_LIB)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
set (PUBLIC_DEPEND_LIBS "")
@@ -108,17 +108,6 @@ ELSE(WIN32)
set (PUBLIC_DEPEND_LIBS ${CMAKE_DL_LIBS})
ENDIF(WIN32)
-# For OpenMP
-# openmp not support well for now on windows
-if (NOT APPLE AND NOT WIN32) # Linux
- find_package(OpenMP)
- if (OPENMP_FOUND)
- add_definitions(-DWITH_OMP)
- set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
- set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
- endif()
-endif()
-
set(CMAKE_INSTALL_PREFIX ${PROJECT_SOURCE_DIR})
set(TOKENIZERS_INSTALL_INCLUDE_DIR ${PROJECT_SOURCE_DIR})
@@ -137,7 +126,7 @@ include_directories(${TOKENIZERS_INSTALL_INCLUDE_DIR})
include(generic)
include(third_party)
-add_subdirectory(faster_tokenizer)
+add_subdirectory(fast_tokenizer)
if(WITH_PYTHON)
@@ -155,18 +144,18 @@ add_custom_target(build_tokenizers_bdist_wheel ALL
DEPENDS copy_python_tokenizers)
endif()
-else(WITH_PYTHON) # Pack faster_tokenizer cpp lib
+else(WITH_PYTHON) # Pack fast_tokenizer cpp lib
-set(CPP_PACKAGE_DIR ${CMAKE_BINARY_DIR}/cpp/faster_tokenizer)
+set(CPP_PACKAGE_DIR ${CMAKE_BINARY_DIR}/cpp/fast_tokenizer)
add_custom_target(build_cpp_package_dir ALL
COMMAND ${CMAKE_COMMAND} -E make_directory ${CPP_PACKAGE_DIR}/lib ${CPP_PACKAGE_DIR}/include ${CPP_PACKAGE_DIR}/third_party/include ${CPP_PACKAGE_DIR}/third_party/lib
DEPENDS core_tokenizers)
# copy cmake
-file(COPY ${PROJECT_SOURCE_DIR}/FasterTokenizer.cmake DESTINATION ${CPP_PACKAGE_DIR}/)
+file(COPY ${PROJECT_SOURCE_DIR}/FastTokenizer.cmake DESTINATION ${CPP_PACKAGE_DIR}/)
# copy headers
-file(COPY ${PROJECT_SOURCE_DIR}/faster_tokenizer/ DESTINATION ${CPP_PACKAGE_DIR}/include/faster_tokenizer/
+file(COPY ${PROJECT_SOURCE_DIR}/fast_tokenizer/ DESTINATION ${CPP_PACKAGE_DIR}/include/fast_tokenizer/
FILES_MATCHING PATTERN "*.h"
PATTERN "test" EXCLUDE
PATTERN "demo" EXCLUDE
@@ -181,7 +170,7 @@ add_custom_target(copy_third_party_headers ALL
# copy library
set(TOKENIZER_CORE_NAME "core_tokenizers")
-set(TOKENIZER_CORE_PATH ${CMAKE_BINARY_DIR}/faster_tokenizer)
+set(TOKENIZER_CORE_PATH ${CMAKE_BINARY_DIR}/fast_tokenizer)
if (WIN32)
set(ICU_DLL_DIR ${CMAKE_BINARY_DIR}/third_party/icu/src/extern_icu/icu4c/bin64)
set(ICU_LIB_DIR ${CMAKE_BINARY_DIR}/third_party/icu/src/extern_icu/icu4c/lib64)
diff --git a/faster_tokenizer/FasterTokenizer.cmake b/fast_tokenizer/FastTokenizer.cmake
similarity index 68%
rename from faster_tokenizer/FasterTokenizer.cmake
rename to fast_tokenizer/FastTokenizer.cmake
index d4f2f4debac2..6d7f8e5f4115 100644
--- a/faster_tokenizer/FasterTokenizer.cmake
+++ b/fast_tokenizer/FastTokenizer.cmake
@@ -16,18 +16,18 @@ endif()
set(LIBRARY_NAME core_tokenizers)
-set(FASTER_TOKENIZER_INCS "")
-list(APPEND FASTER_TOKENIZER_INCS ${CMAKE_CURRENT_LIST_DIR}/include)
-list(APPEND FASTER_TOKENIZER_INCS ${CMAKE_CURRENT_LIST_DIR}/third_party/include)
+set(FAST_TOKENIZER_INCS "")
+list(APPEND FAST_TOKENIZER_INCS ${CMAKE_CURRENT_LIST_DIR}/include)
+list(APPEND FAST_TOKENIZER_INCS ${CMAKE_CURRENT_LIST_DIR}/third_party/include)
-set(FASTER_TOKENIZER_LIBS "")
+set(FAST_TOKENIZER_LIBS "")
find_library(FTLIB ${LIBRARY_NAME} ${CMAKE_CURRENT_LIST_DIR}/lib NO_DEFAULT_PATH)
-list(APPEND FASTER_TOKENIZER_LIBS ${FTLIB})
+list(APPEND FAST_TOKENIZER_LIBS ${FTLIB})
if (WIN32)
find_library(ICUDT icudt ${CMAKE_CURRENT_LIST_DIR}/third_party/lib NO_DEFAULT_PATH)
-list(APPEND FASTER_TOKENIZER_LIBS ${ICUDT})
+list(APPEND FAST_TOKENIZER_LIBS ${ICUDT})
find_library(ICUUC icuuc ${CMAKE_CURRENT_LIST_DIR}/third_party/lib NO_DEFAULT_PATH)
-list(APPEND FASTER_TOKENIZER_LIBS ${ICUUC})
+list(APPEND FAST_TOKENIZER_LIBS ${ICUUC})
endif()
diff --git a/faster_tokenizer/LICENSE b/fast_tokenizer/LICENSE
similarity index 100%
rename from faster_tokenizer/LICENSE
rename to fast_tokenizer/LICENSE
diff --git a/fast_tokenizer/README.md b/fast_tokenizer/README.md
new file mode 100644
index 000000000000..34313a097200
--- /dev/null
+++ b/fast_tokenizer/README.md
@@ -0,0 +1,118 @@
+# FastTokenizer
+
+------------------------------------------------------------------------------------------
+
+
+
+
+
+
+
+
+
+
+
+
+FastTokenizer是一款简单易用、功能强大的跨平台高性能文本预处理库,集成业界多个常用的Tokenizer实现,支持不同NLP场景下的文本预处理功能,如文本分类、阅读理解,序列标注等。结合PaddleNLP Tokenizer模块,为用户在训练、推理阶段提供高效通用的文本预处理能力。
+
+## 特性
+
+- 高性能。由于底层采用C++实现,所以其性能远高于目前常规Python实现的Tokenizer。在文本分类任务上,FastTokenizer对比Python版本Tokenizer加速比最高可达20倍。支持多线程加速多文本批处理分词。默认使用单线程分词。
+- 跨平台。FastTokenizer可在不同的系统平台上使用,目前已支持Windows x64,Linux x64以及MacOS 10.14+平台上使用。
+- 多编程语言支持。FastTokenizer提供在C++、Python语言上开发的能力。
+- 灵活性强。用户可以通过指定不同的FastTokenizer组件定制满足需求的Tokenizer。
+
+## 快速开始
+
+下面将介绍Python版本FastTokenizer的使用方式,C++版本的使用方式可参考[FastTokenizer C++ Demo](./fast_tokenizer/demo/README.md)。
+
+### 环境依赖
+
+- Windows 64位系统
+- Linux x64系统
+- MacOS 10.14+系统(m1芯片的MacOS,需要使用x86_64版本的Anaconda作为python环境方可安装使用)
+- Python 3.6 ~ 3.10
+
+### 安装FastTokenizer
+
+```python
+pip install fast_tokenizer
+```
+
+### FastTokenizer使用示例
+
+- 准备词表
+
+```shell
+# Linux或者Mac用户可直接执行以下命令下载测试的词表,Windows 用户可在浏览器上下载到本地。
+wget https://bj.bcebos.com/paddlenlp/models/transformers/ernie/vocab.txt
+```
+
+- 切词示例
+
+FastTokenizer库内置NLP任务常用的Tokenizer,如ErnieFastTokenizer。下面将展示FastTokenizer的简单用法。
+
+```python
+import fast_tokenizer
+from fast_tokenizer import ErnieFastTokenizer, models
+
+# 0.(可选)设置线程数
+fast_tokenizer.set_thread_num(1)
+# 1. 加载词表
+vocab = models.WordPiece.read_file("ernie_vocab.txt")
+# 2. 实例化ErnieFastTokenizer对象
+fast_tokenizer = ErnieFastTokenizer(vocab)
+# 3. 切词
+output = fast_tokenizer.encode("我爱中国")
+# 4. 输出结果
+print("ids: ", output.ids)
+print("type_ids: ", output.type_ids)
+print("tokens: ", output.tokens)
+print("offsets: ", output.offsets)
+print("attention_mask: ", output.attention_mask)
+```
+
+### FastTokenizer在PaddleNLP Tokenizer模块加速示例
+
+PaddleNLP Tokenizer模块可简单地应用在模型训练以及推理部署的文本预处理阶段,并通过`AutoTokenizer.from_pretrained`方式实例化相应的Tokenizer。其中`AutoTokenizer`默认加载得到的Tokenizer是常规Python实现的Tokenizer,其性能会低于C++实现的FastTokenizer。为了提升PaddleNLP Tokenizer模块性能,目前PaddleNLP Tokenizer模块已经支持使用FastTokenizer作为Tokenizer的后端加速切词阶段。在现有的Tokenizer加载接口中,仅需添加`use_fast=True`这一关键词参数,其余代码保持不变,即可加载Fast版本的Tokenizer,代码示例如下:
+
+```python
+from paddlenlp.transformers import AutoTokenizer
+
+# 默认加载Python版本的Tokenizer
+tokenizer = AutoTokenizer.from_pretrained('ernie-3.0-medium-zh')
+# 打开use_fast开关,可加载Fast版本Tokenizer
+fast_tokenizer = AutoTokenizer.from_pretrained('ernie-3.0-medium-zh', use_fast=True)
+
+text1 = tokenizer('自然语言处理')
+text2 = fast_tokenizer('自然语言处理')
+
+print(text1)
+print(text2)
+```
+
+目前PaddleNLP已支持BERT、ERNIE、TinyBERT以及ERNIE-M 4种Tokenizer的Fast版本,其余模型的Tokenizer暂不支持Fast版本。
+
+## FAQ
+
+Q:我在AutoTokenizer.from_pretrained接口上已经打开`use_fast=True`开关,为什么文本预处理阶段性能上好像没有任何变化?
+
+A:在有三种情况下,打开`use_fast=True`开关可能无法提升性能:
+ 1. 没有安装fast_tokenizer。若在没有安装fast_tokenizer库的情况下打开`use_fast`开关,PaddleNLP会给出以下warning:"Can't find the fast_tokenizer package, please ensure install fast_tokenizer correctly. "。
+
+ 2. 加载的Tokenizer类型暂不支持Fast版本。目前支持4种Tokenizer的Fast版本,分别是BERT、ERNIE、TinyBERT以及ERNIE-M Tokenizer。若加载不支持Fast版本的Tokenizer情况下打开`use_fast`开关,PaddleNLP会给出以下warning:"The tokenizer XXX doesn't have the fast version. Please check the map paddlenlp.transformers.auto.tokenizer.FAST_TOKENIZER_MAPPING_NAMES to see which fast tokenizers are currently supported."
+
+ 3. 待切词文本长度过短(如文本平均长度小于5)。这种情况下切词开销可能不是整个文本预处理的性能瓶颈,导致在使用FastTokenizer后仍无法提升整体性能。
+
+Q:如何使用多线程加速分词?
+
+A:可以通过调用 `fast_tokenizer.set_thread_num(xxx)` 使用多线程进行分词。需要谨慎开启多线程加速分词,在以下场景下可以考虑开启多线程:
+ 1. CPU资源充足。若在推理阶段使用CPU进行推理,开启多线程分词可能会出现资源竞争情况,从而影响推理阶段的性能。
+
+ 2. 文本的批大小较大。若批大小比较小,开启多线程可能不会得到任何加速效果,并且可能会因为线程调度导致延时增长。建议批大小大于4的时候再考虑开启多线程分词。
+
+ 3. 文本长度较长。若文本长度较短,开启多线程可能不会得到任何加速效果,并且可能会因为线程调度导致延时增长。建议文本平均长度大于16的时候再考虑开启多线程分词。
+
+## 相关文档
+
+[FastTokenizer编译指南](docs/compile/README.md)
diff --git a/faster_tokenizer/cmake/ByproductsICU.cmake b/fast_tokenizer/cmake/ByproductsICU.cmake
similarity index 100%
rename from faster_tokenizer/cmake/ByproductsICU.cmake
rename to fast_tokenizer/cmake/ByproductsICU.cmake
diff --git a/faster_tokenizer/cmake/FindNumPy.cmake b/fast_tokenizer/cmake/FindNumPy.cmake
similarity index 100%
rename from faster_tokenizer/cmake/FindNumPy.cmake
rename to fast_tokenizer/cmake/FindNumPy.cmake
diff --git a/faster_tokenizer/cmake/dummy.c.in b/fast_tokenizer/cmake/dummy.c.in
similarity index 100%
rename from faster_tokenizer/cmake/dummy.c.in
rename to fast_tokenizer/cmake/dummy.c.in
diff --git a/faster_tokenizer/cmake/external/dart.cmake b/fast_tokenizer/cmake/external/dart.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/dart.cmake
rename to fast_tokenizer/cmake/external/dart.cmake
diff --git a/faster_tokenizer/cmake/external/gflags.cmake b/fast_tokenizer/cmake/external/gflags.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/gflags.cmake
rename to fast_tokenizer/cmake/external/gflags.cmake
diff --git a/faster_tokenizer/cmake/external/glog.cmake b/fast_tokenizer/cmake/external/glog.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/glog.cmake
rename to fast_tokenizer/cmake/external/glog.cmake
diff --git a/faster_tokenizer/cmake/external/gtest.cmake b/fast_tokenizer/cmake/external/gtest.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/gtest.cmake
rename to fast_tokenizer/cmake/external/gtest.cmake
diff --git a/faster_tokenizer/cmake/external/icu.cmake b/fast_tokenizer/cmake/external/icu.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/icu.cmake
rename to fast_tokenizer/cmake/external/icu.cmake
diff --git a/faster_tokenizer/cmake/external/nlohmann_json.cmake b/fast_tokenizer/cmake/external/nlohmann_json.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/nlohmann_json.cmake
rename to fast_tokenizer/cmake/external/nlohmann_json.cmake
diff --git a/faster_tokenizer/cmake/external/protobuf.cmake b/fast_tokenizer/cmake/external/protobuf.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/protobuf.cmake
rename to fast_tokenizer/cmake/external/protobuf.cmake
diff --git a/faster_tokenizer/cmake/external/pybind11.cmake b/fast_tokenizer/cmake/external/pybind11.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/pybind11.cmake
rename to fast_tokenizer/cmake/external/pybind11.cmake
diff --git a/faster_tokenizer/cmake/external/python.cmake b/fast_tokenizer/cmake/external/python.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/python.cmake
rename to fast_tokenizer/cmake/external/python.cmake
diff --git a/faster_tokenizer/cmake/external/re2.cmake b/fast_tokenizer/cmake/external/re2.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/re2.cmake
rename to fast_tokenizer/cmake/external/re2.cmake
diff --git a/faster_tokenizer/cmake/external/utf8proc.cmake b/fast_tokenizer/cmake/external/utf8proc.cmake
similarity index 100%
rename from faster_tokenizer/cmake/external/utf8proc.cmake
rename to fast_tokenizer/cmake/external/utf8proc.cmake
diff --git a/faster_tokenizer/cmake/generic.cmake b/fast_tokenizer/cmake/generic.cmake
similarity index 100%
rename from faster_tokenizer/cmake/generic.cmake
rename to fast_tokenizer/cmake/generic.cmake
diff --git a/faster_tokenizer/cmake/python_module.cmake b/fast_tokenizer/cmake/python_module.cmake
similarity index 100%
rename from faster_tokenizer/cmake/python_module.cmake
rename to fast_tokenizer/cmake/python_module.cmake
diff --git a/faster_tokenizer/cmake/third_party.cmake b/fast_tokenizer/cmake/third_party.cmake
similarity index 100%
rename from faster_tokenizer/cmake/third_party.cmake
rename to fast_tokenizer/cmake/third_party.cmake
diff --git a/faster_tokenizer/docs/compile/README.md b/fast_tokenizer/docs/compile/README.md
similarity index 52%
rename from faster_tokenizer/docs/compile/README.md
rename to fast_tokenizer/docs/compile/README.md
index d7820884e1f4..b510e6c79a17 100644
--- a/faster_tokenizer/docs/compile/README.md
+++ b/fast_tokenizer/docs/compile/README.md
@@ -1,11 +1,11 @@
-# FasterTokenizer编译指南
+# FastTokenizer编译指南
-本文档说明编译FasterTokenizer C++库、Python库两种编译过程,根据编译的平台参考如下文档
+本文档说明编译FastTokenizer C++库、Python库两种编译过程,根据编译的平台参考如下文档
- [Linux & Mac 编译](./how_to_build_linux_and_mac.md)
- [Windows编译](./how_to_build_windows.md)
-FasterTokenizer使用CMake编译,其中编译过程中,各平台上编译选项如下表所示
+FastTokenizer使用CMake编译,其中编译过程中,各平台上编译选项如下表所示
| 选项 | 作用 | 备注 |
|:---- | :--- | :--- |
diff --git a/faster_tokenizer/docs/compile/how_to_build_linux_and_mac.md b/fast_tokenizer/docs/compile/how_to_build_linux_and_mac.md
similarity index 92%
rename from faster_tokenizer/docs/compile/how_to_build_linux_and_mac.md
rename to fast_tokenizer/docs/compile/how_to_build_linux_and_mac.md
index 5dc820525176..fd856d8984d1 100644
--- a/faster_tokenizer/docs/compile/how_to_build_linux_and_mac.md
+++ b/fast_tokenizer/docs/compile/how_to_build_linux_and_mac.md
@@ -9,7 +9,7 @@
```bash
git clone https://github.com/PaddlePaddle/PaddleNLP.git
-cd PaddleNLP/faster_tokenizer
+cd PaddleNLP/fast_tokenizer
mkdir build & cd build
cmake .. -DWITH_PYTHON=OFF -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
make -j8
@@ -21,7 +21,7 @@ make -j8
```bash
git clone https://github.com/PaddlePaddle/PaddleNLP.git
-cd PaddleNLP/faster_tokenizer
+cd PaddleNLP/fast_tokenizer
mkdir build & cd build
# 设置Python环境
export LD_LIBRARY_PATH=/opt/_internal/cpython-3.6.0/lib/:${LD_LIBRARY_PATH}
diff --git a/faster_tokenizer/docs/compile/how_to_build_windows.md b/fast_tokenizer/docs/compile/how_to_build_windows.md
similarity index 94%
rename from faster_tokenizer/docs/compile/how_to_build_windows.md
rename to fast_tokenizer/docs/compile/how_to_build_windows.md
index b7b73bc7834b..e4ce9dc2af9a 100644
--- a/faster_tokenizer/docs/compile/how_to_build_windows.md
+++ b/fast_tokenizer/docs/compile/how_to_build_windows.md
@@ -13,7 +13,7 @@
```bash
git clone https://github.com/PaddlePaddle/PaddleNLP.git
-cd PaddleNLP/faster_tokenizer
+cd PaddleNLP/fast_tokenizer
mkdir build & cd build
cmake .. -G "Ninja" -DWITH_PYTHON=OFF -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
ninja -j8
@@ -25,7 +25,7 @@ ninja -j8
```bash
git clone https://github.com/PaddlePaddle/PaddleNLP.git
-cd PaddleNLP/faster_tokenizer
+cd PaddleNLP/fast_tokenizer
mkdir build & cd build
# 需要指定Python库
cmake .. -G "Ninja" -DWITH_PYTHON=ON ^
diff --git a/faster_tokenizer/faster_tokenizer/CMakeLists.txt b/fast_tokenizer/fast_tokenizer/CMakeLists.txt
similarity index 90%
rename from faster_tokenizer/faster_tokenizer/CMakeLists.txt
rename to fast_tokenizer/fast_tokenizer/CMakeLists.txt
index cf4abb40395d..61e54191162c 100644
--- a/faster_tokenizer/faster_tokenizer/CMakeLists.txt
+++ b/fast_tokenizer/fast_tokenizer/CMakeLists.txt
@@ -13,7 +13,7 @@ endif()
if (WITH_PYTHON)
add_subdirectory(pybind)
cc_library(core_tokenizers SHARED
- SRCS pybind/pybind.cc tokenizers/ernie_faster_tokenizer.cc
+ SRCS pybind/pybind.cc tokenizers/ernie_fast_tokenizer.cc
DEPS pybind python pybind_normalizers pybind_utils
pybind_pretokenizers pybind_models pybind_decoders
pybind_postprocessors pybind_tokenizers pybind_exception
@@ -33,7 +33,7 @@ endif()
else(WITH_PYTHON)
# add_subdirectory(tokenizers)
cc_library(core_tokenizers SHARED
- SRCS tokenizers/ernie_faster_tokenizer.cc
+ SRCS tokenizers/ernie_fast_tokenizer.cc tokenizers/clip_fast_tokenizer.cc
DEPS normalizers pretokenizers models decoders
postprocessors core added_vocabulary tokenizer json)
diff --git a/faster_tokenizer/faster_tokenizer/core/CMakeLists.txt b/fast_tokenizer/fast_tokenizer/core/CMakeLists.txt
similarity index 55%
rename from faster_tokenizer/faster_tokenizer/core/CMakeLists.txt
rename to fast_tokenizer/fast_tokenizer/core/CMakeLists.txt
index ea831123e90e..6e97ff1a20db 100644
--- a/faster_tokenizer/faster_tokenizer/core/CMakeLists.txt
+++ b/fast_tokenizer/fast_tokenizer/core/CMakeLists.txt
@@ -1,3 +1,4 @@
cc_library(added_vocabulary SRCS added_vocabulary.cc DEPS normalizers pretokenizers json)
-cc_library(tokenizer SRCS tokenizer.cc DEPS added_vocabulary json decoders trie models postprocessors)
-cc_library(core SRCS encoding.cc DEPS json)
+cc_library(base SRCS base.cc)
+cc_library(tokenizer SRCS tokenizer.cc DEPS added_vocabulary json decoders trie models postprocessors base)
+cc_library(core SRCS encoding.cc DEPS json base)
diff --git a/faster_tokenizer/faster_tokenizer/core/added_vocabulary.cc b/fast_tokenizer/fast_tokenizer/core/added_vocabulary.cc
similarity index 98%
rename from faster_tokenizer/faster_tokenizer/core/added_vocabulary.cc
rename to fast_tokenizer/fast_tokenizer/core/added_vocabulary.cc
index 2c6d47dcf0ca..bdb05fa136b8 100644
--- a/faster_tokenizer/faster_tokenizer/core/added_vocabulary.cc
+++ b/fast_tokenizer/fast_tokenizer/core/added_vocabulary.cc
@@ -12,15 +12,15 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
-#include "faster_tokenizer/core/added_vocabulary.h"
-#include "faster_tokenizer/models/model.h"
-#include "faster_tokenizer/normalizers/normalizer.h"
-#include "faster_tokenizer/pretokenizers/pretokenizer.h"
+#include "fast_tokenizer/core/added_vocabulary.h"
+#include "fast_tokenizer/models/model.h"
+#include "fast_tokenizer/normalizers/normalizer.h"
+#include "fast_tokenizer/pretokenizers/pretokenizer.h"
#include "glog/logging.h"
#include "re2/re2.h"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace core {
inline bool StartWithWord(const std::string& sequence) {
@@ -420,5 +420,5 @@ void to_json(nlohmann::json& j, const AddedVocabulary& added_vocab) {
}
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/added_vocabulary.h b/fast_tokenizer/fast_tokenizer/core/added_vocabulary.h
similarity index 92%
rename from faster_tokenizer/faster_tokenizer/core/added_vocabulary.h
rename to fast_tokenizer/fast_tokenizer/core/added_vocabulary.h
index d425d7d6ffe5..a9b26f677818 100644
--- a/faster_tokenizer/faster_tokenizer/core/added_vocabulary.h
+++ b/fast_tokenizer/fast_tokenizer/core/added_vocabulary.h
@@ -18,7 +18,7 @@ limitations under the License. */
#include
#include
-#include "faster_tokenizer/core/base.h"
+#include "fast_tokenizer/core/base.h"
#include "nlohmann/json.hpp"
namespace re2 {
@@ -26,7 +26,7 @@ class RE2;
} // namespace re2
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace normalizers {
class Normalizer;
@@ -52,7 +52,7 @@ bool EndWithWord(const std::string& sequence);
bool StartWithSpace(const std::string& sequence);
bool EndWithSpace(const std::string& sequence);
-class FASTERTOKENIZER_DECL AddedToken {
+class FASTTOKENIZER_DECL AddedToken {
public:
AddedToken();
AddedToken(const std::string& content,
@@ -84,14 +84,14 @@ class FASTERTOKENIZER_DECL AddedToken {
friend struct AddedTokenWithId;
};
-struct FASTERTOKENIZER_DECL AddedTokenWithId {
+struct FASTTOKENIZER_DECL AddedTokenWithId {
AddedToken added_token_;
uint32_t id_;
friend void to_json(nlohmann::json& j, const AddedTokenWithId& added_token);
friend void from_json(const nlohmann::json& j, AddedTokenWithId& added_token);
};
-class FASTERTOKENIZER_DECL AddedVocabulary {
+class FASTTOKENIZER_DECL AddedVocabulary {
public:
AddedVocabulary();
size_t GetLen() const;
@@ -139,15 +139,15 @@ class FASTERTOKENIZER_DECL AddedVocabulary {
};
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
namespace std {
template <>
-class hash {
+class hash {
public:
size_t operator()(
- const paddlenlp::faster_tokenizer::core::AddedToken& added_token) const {
+ const paddlenlp::fast_tokenizer::core::AddedToken& added_token) const {
return std::hash()(added_token.GetContent());
}
};
diff --git a/fast_tokenizer/fast_tokenizer/core/base.cc b/fast_tokenizer/fast_tokenizer/core/base.cc
new file mode 100644
index 000000000000..ec4a37c74f66
--- /dev/null
+++ b/fast_tokenizer/fast_tokenizer/core/base.cc
@@ -0,0 +1,46 @@
+/* Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License. */
+
+#include "fast_tokenizer/core/base.h"
+#include
+
+namespace paddlenlp {
+namespace fast_tokenizer {
+namespace core {
+
+static int fast_tokenizer_thread_num = 1;
+
+void SetThreadNum(int thread_num) { fast_tokenizer_thread_num = thread_num; }
+
+int GetThreadNum() { return fast_tokenizer_thread_num; }
+
+void RunMultiThread(std::function func,
+ size_t batch_size) {
+ int thread_num = GetThreadNum();
+ std::vector vectorOfThread;
+ size_t start_index = 0;
+ size_t step_index = ceil(batch_size / float(thread_num));
+
+ for (size_t thread_index = 0; thread_index < thread_num; thread_index++) {
+ vectorOfThread.emplace_back(std::thread(func, start_index, step_index));
+ start_index = start_index + step_index;
+ }
+ for (size_t thread_index = 0; thread_index < thread_num; thread_index++) {
+ vectorOfThread[thread_index].join();
+ }
+}
+
+} // namespace core
+} // namespace fast_tokenizer
+} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/base.h b/fast_tokenizer/fast_tokenizer/core/base.h
similarity index 92%
rename from faster_tokenizer/faster_tokenizer/core/base.h
rename to fast_tokenizer/fast_tokenizer/core/base.h
index 0fe8e834c56c..21af2d912f7c 100644
--- a/faster_tokenizer/faster_tokenizer/core/base.h
+++ b/fast_tokenizer/fast_tokenizer/core/base.h
@@ -21,8 +21,8 @@ limitations under the License. */
#include
#include
+#include "fast_tokenizer/utils/utils.h"
#include "nlohmann/json.hpp"
-#include "faster_tokenizer/utils/utils.h"
namespace std {
template <>
@@ -36,17 +36,25 @@ struct hash> {
}
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace core {
-enum FASTERTOKENIZER_DECL OffsetType { CHAR, BYTE };
-enum FASTERTOKENIZER_DECL Direction { LEFT, RIGHT };
-enum FASTERTOKENIZER_DECL TruncStrategy {
+enum FASTTOKENIZER_DECL OffsetType { CHAR, BYTE };
+enum FASTTOKENIZER_DECL Direction { LEFT, RIGHT };
+enum FASTTOKENIZER_DECL TruncStrategy {
LONGEST_FIRST,
ONLY_FIRST,
ONLY_SECOND
};
-enum FASTERTOKENIZER_DECL PadStrategy { BATCH_LONGEST, FIXED_SIZE };
+enum FASTTOKENIZER_DECL PadStrategy { BATCH_LONGEST, FIXED_SIZE };
+
+enum FASTTOKENIZER_DECL SplitMode {
+ REMOVED,
+ ISOLATED,
+ MERGED_WITH_PREVIOUS,
+ MERGED_WITH_NEXT,
+ CONTIGUOUS
+};
NLOHMANN_JSON_SERIALIZE_ENUM(OffsetType,
{
@@ -72,7 +80,7 @@ NLOHMANN_JSON_SERIALIZE_ENUM(PadStrategy,
{FIXED_SIZE, "FIXED_SIZE"},
});
-struct FASTERTOKENIZER_DECL TruncMethod {
+struct FASTTOKENIZER_DECL TruncMethod {
Direction direction_;
size_t max_len_;
TruncStrategy strategy_;
@@ -84,7 +92,7 @@ struct FASTERTOKENIZER_DECL TruncMethod {
direction_(RIGHT) {}
};
-struct FASTERTOKENIZER_DECL PadMethod {
+struct FASTTOKENIZER_DECL PadMethod {
PadStrategy strategy_;
Direction direction_;
uint32_t pad_id_;
@@ -160,7 +168,7 @@ inline void to_json(nlohmann::json& j,
}
}
-struct FASTERTOKENIZER_DECL Token {
+struct FASTTOKENIZER_DECL Token {
uint32_t id_;
std::string value_;
Offset offset_;
@@ -169,7 +177,7 @@ struct FASTERTOKENIZER_DECL Token {
: id_(id), value_(value), offset_(offset) {}
};
-struct FASTERTOKENIZER_DECL Merge {
+struct FASTTOKENIZER_DECL Merge {
size_t pos_;
uint32_t rank_;
uint32_t new_id_;
@@ -188,7 +196,7 @@ struct FASTERTOKENIZER_DECL Merge {
}
};
-struct FASTERTOKENIZER_DECL Symbol {
+struct FASTTOKENIZER_DECL Symbol {
uint32_t ch_; // symbol id
int prev_;
int next_;
@@ -208,7 +216,7 @@ struct FASTERTOKENIZER_DECL Symbol {
}
};
-struct FASTERTOKENIZER_DECL BPEWord {
+struct FASTTOKENIZER_DECL BPEWord {
BPEWord() = default;
BPEWord(size_t capacity) { Reserve(capacity); }
void Reserve(size_t capacity) { symbols_.reserve(capacity); }
@@ -358,6 +366,13 @@ struct FASTERTOKENIZER_DECL BPEWord {
std::vector symbols_;
};
+FASTTOKENIZER_DECL void SetThreadNum(int thread_num);
+
+FASTTOKENIZER_DECL int GetThreadNum();
+
+FASTTOKENIZER_DECL void RunMultiThread(std::function func,
+ size_t batch_size);
+
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/encoding.cc b/fast_tokenizer/fast_tokenizer/core/encoding.cc
similarity index 91%
rename from faster_tokenizer/faster_tokenizer/core/encoding.cc
rename to fast_tokenizer/fast_tokenizer/core/encoding.cc
index 980e192abcbc..379d21df931b 100644
--- a/faster_tokenizer/faster_tokenizer/core/encoding.cc
+++ b/fast_tokenizer/fast_tokenizer/core/encoding.cc
@@ -12,19 +12,15 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
-#include "faster_tokenizer/core/encoding.h"
+#include "fast_tokenizer/core/encoding.h"
#include
#include
-#include
+#include
#include
#include "glog/logging.h"
-#ifdef WITH_OMP
-#include
-#endif
-
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace core {
Encoding::Encoding(const std::vector& ids,
@@ -77,7 +73,7 @@ Encoding::Encoding(uint32_t capacity) {
Encoding::Encoding(const std::vector& tokens, uint32_t type_id)
: type_ids_(tokens.size(), type_id),
- words_idx_(tokens.size()),
+ words_idx_(tokens.size(), std::numeric_limits::max()),
attention_mask_(tokens.size(), 1),
special_tokens_mask_(tokens.size(), 0) {
auto length = tokens.size();
@@ -179,10 +175,11 @@ Range Encoding::GetSequenceRange(uint32_t seq_id) const {
}
void Encoding::ProcessTokenWithOffsets(
- std::function process_token_fn) {
+ std::function
+ process_token_fn) {
auto length = GetLen();
for (int i = 0; i < length; ++i) {
- process_token_fn(i, &tokens_[i], &offsets_[i]);
+ process_token_fn(i, tokens_[i], &offsets_[i]);
}
}
@@ -215,7 +212,7 @@ std::vector Encoding::WordIdxToTokensIdx(uint32_t word_idx,
for (uint32_t i = seq_range.first; i < seq_range.second; ++i) {
// -1 is the word index of special token
if (words_idx_[i] > word_idx &&
- words_idx_[i] != static_cast(-1)) {
+ words_idx_[i] != std::numeric_limits::max()) {
break;
}
if (words_idx_[i] == word_idx) {
@@ -434,7 +431,8 @@ void Encoding::Pad(uint32_t target_length,
ids_.insert(ids_.begin(), pad_len, pad_id);
type_ids_.insert(type_ids_.begin(), pad_len, pad_type_id);
tokens_.insert(tokens_.begin(), pad_len, pad_token);
- words_idx_.insert(words_idx_.begin(), pad_len, UINT_MAX);
+ words_idx_.insert(
+ words_idx_.begin(), pad_len, std::numeric_limits::max());
attention_mask_.insert(attention_mask_.begin(), pad_len, 0);
special_tokens_mask_.insert(special_tokens_mask_.begin(), pad_len, 1);
offsets_.insert(offsets_.begin(), pad_len, {0, 0});
@@ -442,7 +440,8 @@ void Encoding::Pad(uint32_t target_length,
ids_.insert(ids_.end(), pad_len, pad_id);
type_ids_.insert(type_ids_.end(), pad_len, pad_type_id);
tokens_.insert(tokens_.end(), pad_len, pad_token);
- words_idx_.insert(words_idx_.end(), pad_len, UINT_MAX);
+ words_idx_.insert(
+ words_idx_.end(), pad_len, std::numeric_limits::max());
attention_mask_.insert(attention_mask_.end(), pad_len, 0);
special_tokens_mask_.insert(special_tokens_mask_.end(), pad_len, 1);
offsets_.insert(offsets_.end(), pad_len, {0, 0});
@@ -460,6 +459,27 @@ Encoding Encoding::Merge(const std::vector& encodings,
return merged_encoding;
}
+void Encoding::SetTypeIds(const std::vector& type_ids) {
+ type_ids_ = type_ids;
+}
+
+bool Encoding::operator==(const Encoding& other) const {
+ if (overflowing_.size() != other.overflowing_.size()) {
+ return false;
+ }
+ for (int i = 0; i < overflowing_.size(); ++i) {
+ if (!(overflowing_[i] == other.overflowing_[i])) {
+ return false;
+ }
+ }
+ return ids_ == other.ids_ && type_ids_ == other.type_ids_ &&
+ tokens_ == other.tokens_ && words_idx_ == other.words_idx_ &&
+ offsets_ == other.offsets_ &&
+ special_tokens_mask_ == other.special_tokens_mask_ &&
+ attention_mask_ == other.attention_mask_ &&
+ sequence_ranges_ == other.sequence_ranges_;
+}
+
std::string Encoding::DebugString() const {
std::ostringstream oss;
oss << "The Encoding content: \n";
@@ -523,7 +543,6 @@ std::string Encoding::DebugString() const {
oss << "{" << iter->first << " : (" << iter->second.first << ", "
<< iter->second.second << ") }, ";
}
- oss << "\n";
return oss.str();
}
@@ -635,17 +654,6 @@ void PadEncodings(std::vector* encodings, const PadMethod& method) {
pad_length += pad_length - pad_length % method.pad_to_multiple_of_;
}
auto batch_size = encodings->size();
-#ifdef WITH_OMP
-#pragma omp parallel for if (batch_size >= 4 && omp_get_max_threads() > 1)
- for (int i = 0; i < batch_size; ++i) {
- auto& encoding = (*encodings)[i];
- encoding.Pad(pad_length,
- method.pad_id_,
- method.pad_token_type_id_,
- method.pad_token_,
- method.direction_);
- }
-#else
auto func = std::bind(&MultiThreadPadEncodings,
encodings,
std::ref(method),
@@ -653,45 +661,9 @@ void PadEncodings(std::vector* encodings, const PadMethod& method) {
std::placeholders::_1,
std::placeholders::_2);
RunMultiThread(func, batch_size);
-#endif
}
-int GetThreadNum(size_t batch_size) {
- char* env_var = std::getenv("OMP_NUM_THREADS");
- int thread_num = std::atoi(env_var);
- if (batch_size <= 0) {
- thread_num = 1;
- VLOG(3) << "batch_size <=0, we set OMP_NUM_THREADS = 1";
- } else {
- int best_num = ceil(batch_size / 4.0);
- if (thread_num > best_num) {
- thread_num = best_num;
- VLOG(3) << "OMP_NUM_THREADS > batch_size/4, we set OMP_NUM_THREADS = "
- "batch_size/4";
- } else if (thread_num == 0) {
- thread_num = best_num;
- VLOG(3) << "OMP_NUM_THREADS == 0, we set OMP_NUM_THREADS = batch_size/4";
- }
- }
- return thread_num;
-}
-
-void RunMultiThread(std::function func,
- size_t batch_size) {
- int thread_num = GetThreadNum(batch_size);
- std::vector vectorOfThread;
- size_t start_index = 0;
- size_t step_index = ceil(batch_size / float(thread_num));
-
- for (size_t thread_index = 0; thread_index < thread_num; thread_index++) {
- vectorOfThread.emplace_back(std::thread(func, start_index, step_index));
- start_index = start_index + step_index;
- }
- for (size_t thread_index = 0; thread_index < thread_num; thread_index++) {
- vectorOfThread[thread_index].join();
- }
-}
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/encoding.h b/fast_tokenizer/fast_tokenizer/core/encoding.h
similarity index 85%
rename from faster_tokenizer/faster_tokenizer/core/encoding.h
rename to fast_tokenizer/fast_tokenizer/core/encoding.h
index 12a4bb708635..5a9d3a41b714 100644
--- a/faster_tokenizer/faster_tokenizer/core/encoding.h
+++ b/fast_tokenizer/fast_tokenizer/core/encoding.h
@@ -18,20 +18,19 @@ limitations under the License. */
#include
#include
#include
-#include "faster_tokenizer/core/base.h"
-#include "faster_tokenizer/utils/utils.h"
+#include "fast_tokenizer/core/base.h"
+#include "fast_tokenizer/utils/utils.h"
#include
#include
#include
#include
-using namespace std;
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace core {
-class FASTERTOKENIZER_DECL Encoding {
+class FASTTOKENIZER_DECL Encoding {
public:
Encoding() = default;
Encoding(const std::vector& ids,
@@ -82,7 +81,8 @@ class FASTERTOKENIZER_DECL Encoding {
Range GetSequenceRange(uint32_t seq_id) const;
void ProcessTokenWithOffsets(
- std::function process_token_fn);
+ std::function
+ process_token_fn);
// token_idx: The index of token in the sequence
std::vector TokenIdxToSequenceIds(uint32_t token_idx) const;
@@ -109,6 +109,8 @@ class FASTERTOKENIZER_DECL Encoding {
static Encoding Merge(const std::vector& encodings,
bool growing_offsets);
std::string DebugString() const;
+ void SetTypeIds(const std::vector& type_ids);
+ bool operator==(const Encoding& other) const;
private:
std::vector ids_;
@@ -122,16 +124,12 @@ class FASTERTOKENIZER_DECL Encoding {
std::unordered_map sequence_ranges_;
};
-bool FASTERTOKENIZER_DECL TruncateEncodings(Encoding* encoding,
- Encoding* pair_encoding,
- const TruncMethod& method);
-void FASTERTOKENIZER_DECL PadEncodings(std::vector* encoding,
- const PadMethod& method);
+bool FASTTOKENIZER_DECL TruncateEncodings(Encoding* encoding,
+ Encoding* pair_encoding,
+ const TruncMethod& method);
+void FASTTOKENIZER_DECL PadEncodings(std::vector* encoding,
+ const PadMethod& method);
-int FASTERTOKENIZER_DECL GetThreadNum(size_t batch_size);
-
-void FASTERTOKENIZER_DECL
-RunMultiThread(std::function func, size_t batch_size);
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/tokenizer.cc b/fast_tokenizer/fast_tokenizer/core/tokenizer.cc
similarity index 81%
rename from faster_tokenizer/faster_tokenizer/core/tokenizer.cc
rename to fast_tokenizer/fast_tokenizer/core/tokenizer.cc
index 1b6399c4aedf..f7689f6b76ac 100644
--- a/faster_tokenizer/faster_tokenizer/core/tokenizer.cc
+++ b/fast_tokenizer/fast_tokenizer/core/tokenizer.cc
@@ -15,24 +15,19 @@ limitations under the License. */
#include
#include "glog/logging.h"
-#include "faster_tokenizer/core/added_vocabulary.h"
-#include "faster_tokenizer/core/base.h"
-#include "faster_tokenizer/core/encoding.h"
-#include "faster_tokenizer/core/tokenizer.h"
+#include "fast_tokenizer/core/added_vocabulary.h"
+#include "fast_tokenizer/core/base.h"
+#include "fast_tokenizer/core/encoding.h"
+#include "fast_tokenizer/core/tokenizer.h"
-#include "faster_tokenizer/decoders/decoders.h"
-#include "faster_tokenizer/models/models.h"
-#include "faster_tokenizer/normalizers/normalizers.h"
-#include "faster_tokenizer/postprocessors/postprocessors.h"
-#include "faster_tokenizer/pretokenizers/pretokenizers.h"
-
-
-#ifdef WITH_OMP
-#include
-#endif
+#include "fast_tokenizer/decoders/decoders.h"
+#include "fast_tokenizer/models/models.h"
+#include "fast_tokenizer/normalizers/normalizers.h"
+#include "fast_tokenizer/postprocessors/postprocessors.h"
+#include "fast_tokenizer/pretokenizers/pretokenizers.h"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace core {
normalizers::Normalizer* Tokenizer::GetNormalizerPtr() const {
@@ -249,118 +244,116 @@ void Tokenizer::EncodePairStrings(const EncodeInput& encode_input,
}
}
+void Tokenizer::EncodePairStrings(const std::string& text,
+ const std::string& text_pair,
+ Encoding* encodings,
+ bool add_special_tokens) const {
+ Encoding encoding, pair_encoding;
+ EncodeSingleString(text, 0, OffsetType::CHAR, &encoding);
+ EncodeSingleString(text_pair, 1, OffsetType::CHAR, &pair_encoding);
+ PostProcess(&encoding, &pair_encoding, add_special_tokens, encodings);
+}
+
void Tokenizer::MultiThreadEncodeBatchStrings(
- const std::vector& batch_encode_input,
+ const std::vector& texts,
+ const std::vector& text_pairs,
std::vector* encodings,
bool add_special_tokens,
size_t start_index,
size_t step_index) const {
- auto batch_size = batch_encode_input.size();
+ if (texts.size() != text_pairs.size()) {
+ throw std::runtime_error(
+ "The size of text must equal to the size of text_pair");
+ }
+ auto batch_size = texts.size();
size_t end_index = start_index + step_index;
if (end_index > batch_size) end_index = batch_size;
for (size_t i = start_index; i < end_index; ++i) {
EncodePairStrings(
- batch_encode_input[i], &(*encodings)[i], add_special_tokens);
+ texts[i], text_pairs[i], &(*encodings)[i], add_special_tokens);
}
}
-void Tokenizer::EncodeBatchStrings(
+void Tokenizer::MultiThreadEncodeBatchStrings(
const std::vector& batch_encode_input,
std::vector* encodings,
- bool add_special_tokens) const {
+ bool add_special_tokens,
+ size_t start_index,
+ size_t step_index) const {
auto batch_size = batch_encode_input.size();
- encodings->resize(batch_size);
-
-#ifdef WITH_OMP
-// (TODO:zhoushunjie): Simply use the batch size to estimate the workload of
-// tokenization.
-// Use workload to determine whether create omp threads. Need to optimize the
-// workload estimation.
-#pragma omp parallel for if (batch_size >= 4 && omp_get_max_threads() > 1)
- for (int i = 0; i < batch_size; ++i) {
+ size_t end_index = start_index + step_index;
+ if (end_index > batch_size) end_index = batch_size;
+ for (size_t i = start_index; i < end_index; ++i) {
EncodePairStrings(
batch_encode_input[i], &(*encodings)[i], add_special_tokens);
}
-#else
- auto func = std::bind(&Tokenizer::MultiThreadEncodeBatchStrings,
- this,
- std::ref(batch_encode_input),
- encodings,
- add_special_tokens,
- std::placeholders::_1,
- std::placeholders::_2);
- RunMultiThread(func, batch_size);
-#endif
-
- if (use_padding_) {
- PadEncodings(encodings, pad_method_);
- }
-}
-
-void Tokenizer::EncodePairStringsCharOffsets(const EncodeInput& encode_input,
- Encoding* encodings,
- bool add_special_tokens) const {
- const auto& input_string = paddlenlp::get_if(&encode_input);
- const auto& input_string_pair =
- paddlenlp::get_if>(&encode_input);
- Encoding encoding;
- Encoding pair_encoding;
- if (input_string != nullptr) {
- EncodeSingleString(*input_string, 0, OffsetType::CHAR, &encoding);
- } else {
- EncodeSingleString(
- input_string_pair->first, 0, OffsetType::CHAR, &encoding);
- EncodeSingleString(
- input_string_pair->second, 1, OffsetType::CHAR, &pair_encoding);
- }
- PostProcess(&encoding, &pair_encoding, add_special_tokens, encodings);
}
-void Tokenizer::MultiThreadEncodeBatchStringsCharOffsets(
- const std::vector& batch_encode_input,
+void Tokenizer::MultiThreadEncodeBatchStrings(
+ const std::vector& texts,
std::vector* encodings,
bool add_special_tokens,
size_t start_index,
size_t step_index) const {
- auto batch_size = batch_encode_input.size();
+ auto batch_size = texts.size();
size_t end_index = start_index + step_index;
if (end_index > batch_size) end_index = batch_size;
for (size_t i = start_index; i < end_index; ++i) {
- Encoding encoding;
- EncodePairStringsCharOffsets(
- batch_encode_input[i], &encoding, add_special_tokens);
- (*encodings)[i] = std::move(encoding);
+ EncodePairStrings(texts[i], &(*encodings)[i], add_special_tokens);
}
}
-void Tokenizer::EncodeBatchStringsCharOffsets(
+void Tokenizer::EncodeBatchStrings(
const std::vector& batch_encode_input,
std::vector* encodings,
bool add_special_tokens) const {
auto batch_size = batch_encode_input.size();
encodings->resize(batch_size);
-#ifdef WITH_OMP
-// (TODO:zhoushunjie): Simply use the batch size to estimate the workload of
-// tokenization.
-// Use workload to determine whether create omp threads. Need to optimize the
-// workload estimation.
-#pragma omp parallel for if (batch_size >= 4 && omp_get_max_threads() > 1)
- for (int i = 0; i < batch_size; ++i) {
- Encoding encoding;
- EncodePairStringsCharOffsets(
- batch_encode_input[i], &encoding, add_special_tokens);
- (*encodings)[i] = std::move(encoding);
+ auto func = [&](size_t start_index, size_t step_index) {
+ MultiThreadEncodeBatchStrings(batch_encode_input,
+ encodings,
+ add_special_tokens,
+ start_index,
+ step_index);
+ };
+ RunMultiThread(func, batch_size);
+
+ if (use_padding_) {
+ PadEncodings(encodings, pad_method_);
}
-#else
- auto func = std::bind(&Tokenizer::MultiThreadEncodeBatchStringsCharOffsets,
- this,
- std::ref(batch_encode_input),
- encodings,
- add_special_tokens,
- std::placeholders::_1,
- std::placeholders::_2);
+}
+
+void Tokenizer::EncodeBatchStrings(const std::vector& texts,
+ std::vector* encodings,
+ bool add_special_tokens) const {
+ auto batch_size = texts.size();
+ encodings->resize(batch_size);
+ auto func = [&](size_t start_index, size_t step_index) {
+ MultiThreadEncodeBatchStrings(
+ texts, encodings, add_special_tokens, start_index, step_index);
+ };
+ RunMultiThread(func, batch_size);
+
+ if (use_padding_) {
+ PadEncodings(encodings, pad_method_);
+ }
+}
+
+void Tokenizer::EncodeBatchStrings(const std::vector& texts,
+ const std::vector& text_pairs,
+ std::vector* encodings,
+ bool add_special_tokens) const {
+ auto batch_size = texts.size();
+ encodings->resize(batch_size);
+ auto func = [&](size_t start_index, size_t step_index) {
+ MultiThreadEncodeBatchStrings(texts,
+ text_pairs,
+ encodings,
+ add_special_tokens,
+ start_index,
+ step_index);
+ };
RunMultiThread(func, batch_size);
-#endif
if (use_padding_) {
PadEncodings(encodings, pad_method_);
@@ -398,6 +391,7 @@ Encoding Tokenizer::EncodeTextToEncoding(const std::vector& word_idx,
DoTokenize(&pretokenized, type_id, word_idx, offset_type, &encoding);
return encoding;
}
+
const AddedVocabulary& Tokenizer::GetAddedVocabulary() const {
return added_vocabulary_;
}
@@ -479,26 +473,11 @@ void Tokenizer::DecodeBatch(
bool skip_special_tokens) const {
auto batch_size = batch_token_ids.size();
results->resize(batch_size);
-#ifdef WITH_OMP
-// (TODO:zhoushunjie): Simply use the batch size to estimate the workload of
-// tokenization.
-// Use workload to determine whether create omp threads. Need to optimize the
-// workload estimation.
-#pragma omp parallel for if (batch_token_ids.size() >= 4 && \
- omp_get_num_threads() > 1)
- for (int i = 0; i < batch_token_ids.size(); ++i) {
- Decode(batch_token_ids[i], &(*results)[i], skip_special_tokens);
- }
-#else
- auto func = std::bind(&Tokenizer::MultiThreadDecodeBatch,
- this,
- std::ref(batch_token_ids),
- results,
- skip_special_tokens,
- std::placeholders::_1,
- std::placeholders::_2);
+ auto func = [&](size_t start_index, size_t step_index) {
+ MultiThreadDecodeBatch(
+ batch_token_ids, results, skip_special_tokens, start_index, step_index);
+ };
RunMultiThread(func, batch_size);
-#endif
}
bool Tokenizer::GetUseTruncation() const { return use_truncation_; }
@@ -591,6 +570,14 @@ void to_json(nlohmann::json& j, const Tokenizer& tokenizer) {
typeid(pretokenizers::SequencePreTokenizer)) {
j["pretokenizer"] = *dynamic_cast(
tokenizer.pretokenizer_.get());
+ } else if (typeid(*tokenizer.pretokenizer_.get()) ==
+ typeid(pretokenizers::ByteLevelPreTokenizer)) {
+ j["pretokenizer"] = *dynamic_cast(
+ tokenizer.pretokenizer_.get());
+ } else if (typeid(*tokenizer.pretokenizer_.get()) ==
+ typeid(pretokenizers::SplitPreTokenizer)) {
+ j["pretokenizer"] = *dynamic_cast(
+ tokenizer.pretokenizer_.get());
}
}
@@ -599,9 +586,9 @@ void to_json(nlohmann::json& j, const Tokenizer& tokenizer) {
if (typeid(*tokenizer.model_.get()) == typeid(models::WordPiece)) {
j["model"] = *dynamic_cast(tokenizer.model_.get());
} else if (typeid(*tokenizer.model_.get()) ==
- typeid(models::FasterWordPiece)) {
+ typeid(models::FastWordPiece)) {
j["model"] =
- *dynamic_cast(tokenizer.model_.get());
+ *dynamic_cast(tokenizer.model_.get());
} else if (typeid(*tokenizer.model_.get()) == typeid(models::BPE)) {
j["model"] = *dynamic_cast(tokenizer.model_.get());
} else if (typeid(*tokenizer.model_.get()) == typeid(models::Unigram)) {
@@ -620,6 +607,15 @@ void to_json(nlohmann::json& j, const Tokenizer& tokenizer) {
j["postprocessor"] =
*dynamic_cast(
tokenizer.post_processor_.get());
+ } else if (typeid(*tokenizer.post_processor_.get()) ==
+ typeid(postprocessors::RobertaPostProcessor)) {
+ j["postprocessor"] = *dynamic_cast(
+ tokenizer.post_processor_.get());
+ } else if (typeid(*tokenizer.post_processor_.get()) ==
+ typeid(postprocessors::ByteLevelPostProcessor)) {
+ j["postprocessor"] =
+ *dynamic_cast(
+ tokenizer.post_processor_.get());
}
}
@@ -705,6 +701,14 @@ void from_json(const nlohmann::json& j, Tokenizer& tokenizer) {
pretokenizers::SequencePreTokenizer sequence_pretokenizer;
pretokenizer.get_to(sequence_pretokenizer);
tokenizer.SetPreTokenizer(sequence_pretokenizer);
+ } else if (pretokenizer.at("type") == "ByteLevelPreTokenizer") {
+ pretokenizers::ByteLevelPreTokenizer byte_pretokenizer;
+ pretokenizer.get_to(byte_pretokenizer);
+ tokenizer.SetPreTokenizer(byte_pretokenizer);
+ } else if (pretokenizer.at("type") == "SplitPreTokenizer") {
+ pretokenizers::SplitPreTokenizer split_pretokenizer;
+ pretokenizer.get_to(split_pretokenizer);
+ tokenizer.SetPreTokenizer(split_pretokenizer);
}
}
@@ -715,8 +719,8 @@ void from_json(const nlohmann::json& j, Tokenizer& tokenizer) {
models::WordPiece wordpiece;
model.get_to(wordpiece);
tokenizer.SetModel(wordpiece);
- } else if (model.at("type") == "FasterWordPiece") {
- models::FasterWordPiece wordpiece;
+ } else if (model.at("type") == "FastWordPiece") {
+ models::FastWordPiece wordpiece;
model.get_to(wordpiece);
tokenizer.SetModel(wordpiece);
} else if (model.at("type") == "BPE") {
@@ -741,6 +745,14 @@ void from_json(const nlohmann::json& j, Tokenizer& tokenizer) {
postprocessors::TemplatePostProcessor template_postprocessor;
post_processor.get_to(template_postprocessor);
tokenizer.SetPostProcessor(template_postprocessor);
+ } else if (post_processor.at("type") == "RobertaPostProcessor") {
+ postprocessors::RobertaPostProcessor roberta_postprocessor;
+ post_processor.get_to(roberta_postprocessor);
+ tokenizer.SetPostProcessor(roberta_postprocessor);
+ } else if (post_processor.at("type") == "ByteLevelPostProcessor") {
+ postprocessors::ByteLevelPostProcessor byte_level_postprocessor;
+ post_processor.get_to(byte_level_postprocessor);
+ tokenizer.SetPostProcessor(byte_level_postprocessor);
}
}
@@ -810,12 +822,16 @@ template void Tokenizer::SetPreTokenizer(
const pretokenizers::MetaSpacePreTokenizer&);
template void Tokenizer::SetPreTokenizer(
const pretokenizers::SequencePreTokenizer&);
+template void Tokenizer::SetPreTokenizer(
+ const pretokenizers::ByteLevelPreTokenizer&);
+template void Tokenizer::SetPreTokenizer(
+ const pretokenizers::SplitPreTokenizer&);
// Instantiate models
template Tokenizer::Tokenizer(const models::WordPiece&);
template void Tokenizer::SetModel(const models::WordPiece&);
-template Tokenizer::Tokenizer(const models::FasterWordPiece&);
-template void Tokenizer::SetModel(const models::FasterWordPiece&);
+template Tokenizer::Tokenizer(const models::FastWordPiece&);
+template void Tokenizer::SetModel(const models::FastWordPiece&);
template Tokenizer::Tokenizer(const models::BPE&);
template void Tokenizer::SetModel(const models::BPE&);
template Tokenizer::Tokenizer(const models::Unigram&);
@@ -826,9 +842,13 @@ template void Tokenizer::SetPostProcessor(
const postprocessors::BertPostProcessor&);
template void Tokenizer::SetPostProcessor(
const postprocessors::TemplatePostProcessor&);
+template void Tokenizer::SetPostProcessor(
+ const postprocessors::RobertaPostProcessor&);
+template void Tokenizer::SetPostProcessor(
+ const postprocessors::ByteLevelPostProcessor&);
// Instantiate Decoder
template void Tokenizer::SetDecoder(const decoders::WordPiece& decoder);
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/core/tokenizer.h b/fast_tokenizer/fast_tokenizer/core/tokenizer.h
similarity index 81%
rename from faster_tokenizer/faster_tokenizer/core/tokenizer.h
rename to fast_tokenizer/fast_tokenizer/core/tokenizer.h
index d709cc5a5c6e..71fdf6099f64 100644
--- a/faster_tokenizer/faster_tokenizer/core/tokenizer.h
+++ b/fast_tokenizer/fast_tokenizer/core/tokenizer.h
@@ -16,14 +16,14 @@ limitations under the License. */
#include // For shared_ptr
#include
-#include "faster_tokenizer/core/added_vocabulary.h"
-#include "faster_tokenizer/core/base.h"
-#include "faster_tokenizer/utils/utils.h"
-#include "faster_tokenizer/utils/variant.h"
+#include "fast_tokenizer/core/added_vocabulary.h"
+#include "fast_tokenizer/core/base.h"
+#include "fast_tokenizer/utils/utils.h"
+#include "fast_tokenizer/utils/variant.h"
#include "nlohmann/json.hpp"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace normalizers {
@@ -60,7 +60,7 @@ using InputString = paddlenlp::variant>;
using EncodeInput =
paddlenlp::variant>;
-class FASTERTOKENIZER_DECL Tokenizer {
+class FASTTOKENIZER_DECL Tokenizer {
public:
Tokenizer()
: model_(nullptr),
@@ -149,16 +149,17 @@ class FASTERTOKENIZER_DECL Tokenizer {
uint32_t type_id,
OffsetType offset_type,
Encoding* encodings) const;
- void EncodePairStrings(const EncodeInput& encode_input,
- Encoding* encodings,
- bool add_special_tokens = true) const;
- void EncodePairStringsCharOffsets(const EncodeInput& encode_input,
- Encoding* encodings,
- bool add_special_tokens = true) const;
void PostProcess(Encoding* encoding,
Encoding* pair_encoding,
bool add_special_tokens,
Encoding* result_encoding) const;
+ void EncodePairStrings(const EncodeInput& encode_input,
+ Encoding* encodings,
+ bool add_special_tokens = true) const;
+ void EncodePairStrings(const std::string& text,
+ const std::string& text_pair,
+ Encoding* encodings,
+ bool add_special_tokens = true) const;
void MultiThreadEncodeBatchStrings(
const std::vector& batch_encode_input,
@@ -166,22 +167,30 @@ class FASTERTOKENIZER_DECL Tokenizer {
bool add_special_tokens,
size_t start_index,
size_t step_index) const;
+ // Tokenize the unpretokenized text.
+ void MultiThreadEncodeBatchStrings(const std::vector& texts,
+ std::vector* encodings,
+ bool add_special_tokens,
+ size_t start_index,
+ size_t step_index) const;
+ void MultiThreadEncodeBatchStrings(const std::vector& texts,
+ const std::vector& text_pairs,
+ std::vector* encodings,
+ bool add_special_tokens,
+ size_t start_index,
+ size_t step_index) const;
void EncodeBatchStrings(const std::vector& batch_encode_input,
std::vector* encodings,
bool add_special_tokens = true) const;
-
- void MultiThreadEncodeBatchStringsCharOffsets(
- const std::vector& batch_encode_input,
- std::vector* encodings,
- bool add_special_tokens,
- size_t start_index,
- size_t step_index) const;
-
- void EncodeBatchStringsCharOffsets(
- const std::vector& batch_encode_input,
- std::vector* encodings,
- bool add_special_tokens = true) const;
+ // Tokenize the unpretokenized text.
+ void EncodeBatchStrings(const std::vector& texts,
+ std::vector* encodings,
+ bool add_special_tokens = true) const;
+ void EncodeBatchStrings(const std::vector& texts,
+ const std::vector& text_pairs,
+ std::vector* encodings,
+ bool add_special_tokens = true) const;
// Encode single text which is already pretokenized.
void EncodeSingleText(const std::vector& pretokenized_texts,
@@ -241,5 +250,5 @@ class FASTERTOKENIZER_DECL Tokenizer {
};
} // namespace core
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/decoders/CMakeLists.txt b/fast_tokenizer/fast_tokenizer/decoders/CMakeLists.txt
similarity index 100%
rename from faster_tokenizer/faster_tokenizer/decoders/CMakeLists.txt
rename to fast_tokenizer/fast_tokenizer/decoders/CMakeLists.txt
diff --git a/faster_tokenizer/faster_tokenizer/decoders/decoder.h b/fast_tokenizer/fast_tokenizer/decoders/decoder.h
similarity index 85%
rename from faster_tokenizer/faster_tokenizer/decoders/decoder.h
rename to fast_tokenizer/fast_tokenizer/decoders/decoder.h
index 081e6693f13d..7969e22d11f7 100644
--- a/faster_tokenizer/faster_tokenizer/decoders/decoder.h
+++ b/fast_tokenizer/fast_tokenizer/decoders/decoder.h
@@ -16,17 +16,17 @@ limitations under the License. */
#include
#include
-#include "faster_tokenizer/utils/utils.h"
+#include "fast_tokenizer/utils/utils.h"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace decoders {
-struct FASTERTOKENIZER_DECL Decoder {
+struct FASTTOKENIZER_DECL Decoder {
virtual void operator()(const std::vector tokens,
std::string* result) const = 0;
};
} // namespace decoders
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/decoders/decoders.h b/fast_tokenizer/fast_tokenizer/decoders/decoders.h
similarity index 86%
rename from faster_tokenizer/faster_tokenizer/decoders/decoders.h
rename to fast_tokenizer/fast_tokenizer/decoders/decoders.h
index 29f8111df4ca..efc72779de9f 100644
--- a/faster_tokenizer/faster_tokenizer/decoders/decoders.h
+++ b/fast_tokenizer/fast_tokenizer/decoders/decoders.h
@@ -14,5 +14,5 @@ limitations under the License. */
#pragma once
-#include "faster_tokenizer/decoders/decoder.h"
-#include "faster_tokenizer/decoders/wordpiece.h"
\ No newline at end of file
+#include "fast_tokenizer/decoders/decoder.h"
+#include "fast_tokenizer/decoders/wordpiece.h"
\ No newline at end of file
diff --git a/faster_tokenizer/faster_tokenizer/decoders/wordpiece.cc b/fast_tokenizer/fast_tokenizer/decoders/wordpiece.cc
similarity index 93%
rename from faster_tokenizer/faster_tokenizer/decoders/wordpiece.cc
rename to fast_tokenizer/fast_tokenizer/decoders/wordpiece.cc
index 385457c15292..e81f1562d242 100644
--- a/faster_tokenizer/faster_tokenizer/decoders/wordpiece.cc
+++ b/fast_tokenizer/fast_tokenizer/decoders/wordpiece.cc
@@ -12,11 +12,11 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
-#include "faster_tokenizer/decoders/wordpiece.h"
-#include "faster_tokenizer/utils/utils.h"
+#include "fast_tokenizer/decoders/wordpiece.h"
+#include "fast_tokenizer/utils/utils.h"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace decoders {
WordPiece::WordPiece(const std::string prefix, bool cleanup)
@@ -65,5 +65,5 @@ void from_json(const nlohmann::json& j, WordPiece& decoder) {
}
} // namespace decoders
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/faster_tokenizer/faster_tokenizer/decoders/wordpiece.h b/fast_tokenizer/fast_tokenizer/decoders/wordpiece.h
similarity index 84%
rename from faster_tokenizer/faster_tokenizer/decoders/wordpiece.h
rename to fast_tokenizer/fast_tokenizer/decoders/wordpiece.h
index 69d93675eba7..1f41b3f8b5dc 100644
--- a/faster_tokenizer/faster_tokenizer/decoders/wordpiece.h
+++ b/fast_tokenizer/fast_tokenizer/decoders/wordpiece.h
@@ -14,15 +14,15 @@ limitations under the License. */
#pragma once
-#include "faster_tokenizer/decoders/decoder.h"
-#include "faster_tokenizer/utils/utils.h"
+#include "fast_tokenizer/decoders/decoder.h"
+#include "fast_tokenizer/utils/utils.h"
#include "nlohmann/json.hpp"
namespace paddlenlp {
-namespace faster_tokenizer {
+namespace fast_tokenizer {
namespace decoders {
-struct FASTERTOKENIZER_DECL WordPiece : public Decoder {
+struct FASTTOKENIZER_DECL WordPiece : public Decoder {
virtual void operator()(const std::vector tokens,
std::string* result) const;
@@ -38,5 +38,5 @@ struct FASTERTOKENIZER_DECL WordPiece : public Decoder {
};
} // namespace decoders
-} // namespace faster_tokenizer
+} // namespace fast_tokenizer
} // namespace paddlenlp
diff --git a/fast_tokenizer/fast_tokenizer/demo/CMakeLists.txt b/fast_tokenizer/fast_tokenizer/demo/CMakeLists.txt
new file mode 100644
index 000000000000..fc0669fc128c
--- /dev/null
+++ b/fast_tokenizer/fast_tokenizer/demo/CMakeLists.txt
@@ -0,0 +1,41 @@
+cmake_minimum_required(VERSION 3.10)
+project(cpp_fast_tokenizer_demo CXX C)
+
+option(FAST_TOKENIZER_INSTALL_DIR "Path of downloaded fast_tokenizer sdk.")
+
+# Download ernie vocab for demo
+set(ERNIE_VOCAB_PATH ${CMAKE_CURRENT_BINARY_DIR}/ernie_vocab.txt)
+if (EXISTS ${ERNIE_VOCAB_PATH})
+ message(STATUS "The ${ERNIE_VOCAB_PATH} exists already.")
+else()
+ file(DOWNLOAD "https://bj.bcebos.com/paddlenlp/models/transformers/ernie/vocab.txt" ${ERNIE_VOCAB_PATH} SHOW_PROGRESS)
+ message(STATUS "Already download the vocab.txt of ernie to ${CMAKE_CURRENT_BINARY_DIR} for demo.")
+endif()
+
+# Download clip vocab and merge files
+set(CLIP_VOCAB_PATH ${CMAKE_CURRENT_BINARY_DIR}/clip_vocab.json)
+set(CLIP_MERGES_PATH ${CMAKE_CURRENT_BINARY_DIR}/clip_merges.txt)
+
+if (EXISTS ${CLIP_VOCAB_PATH})
+ message("The ${CLIP_VOCAB_PATH} exists already.")
+else()
+ file(DOWNLOAD "http://bj.bcebos.com/paddlenlp/models/community/openai/clip-vit-large-patch14/vocab.json" ${CLIP_VOCAB_PATH} SHOW_PROGRESS)
+ message("Already download the vocab.json of clip to ${CMAKE_CURRENT_BINARY_DIR} for test.")
+endif()
+
+if (EXISTS ${CLIP_MERGES_PATH})
+ message("The ${CLIP_MERGES_PATH} exists already.")
+else()
+ file(DOWNLOAD "http://bj.bcebos.com/paddlenlp/models/community/openai/clip-vit-large-patch14/merges.txt" ${CLIP_MERGES_PATH} SHOW_PROGRESS)
+ message("Already download the merges.txt of clip to ${CMAKE_CURRENT_BINARY_DIR} for test.")
+endif()
+
+# Get FAST_TOKENIZER_INCS and FAST_TOKENIZER_LIBS
+include(${FAST_TOKENIZER_INSTALL_DIR}/FastTokenizer.cmake)
+
+include_directories(${FAST_TOKENIZER_INCS})
+
+add_executable(ernie_fast_tokenizer_demo ${PROJECT_SOURCE_DIR}/ernie_fast_tokenizer_demo.cc)
+add_executable(clip_fast_tokenizer_demo ${PROJECT_SOURCE_DIR}/clip_fast_tokenizer_demo.cc)
+target_link_libraries(ernie_fast_tokenizer_demo ${FAST_TOKENIZER_LIBS})
+target_link_libraries(clip_fast_tokenizer_demo ${FAST_TOKENIZER_LIBS})
diff --git a/faster_tokenizer/faster_tokenizer/demo/README.md b/fast_tokenizer/fast_tokenizer/demo/README.md
similarity index 60%
rename from faster_tokenizer/faster_tokenizer/demo/README.md
rename to fast_tokenizer/fast_tokenizer/demo/README.md
index 1b2c1c7110df..e9c745199376 100644
--- a/faster_tokenizer/faster_tokenizer/demo/README.md
+++ b/fast_tokenizer/fast_tokenizer/demo/README.md
@@ -1,15 +1,15 @@
-# FasterTokenizer C++ Demo
+# FastTokenizer C++ Demo
## 1. 快速安装
-当前版本FasterTokenizer C++库支持不同的操作系统以及硬件平台,并为以下平台提供预编译包:
+当前版本FastTokenizer C++库支持不同的操作系统以及硬件平台,并为以下平台提供预编译包:
|系统|下载地址|
|---|---|
-|Linux-x64| [faster_tokenizer-linux-x64-dev.tgz](https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-linux-x64-dev.tgz) |
-|Linux-aarch64| [faster_tokenizer-linux-aarch64-dev.tgz](https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-linux-aarch64-dev.tgz) |
-|Windows| [faster_tokenizer-win-x64-dev.zip](https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-win-x64-dev.zip) |
-|MacOS-x64| [faster_tokenizer-osx-x86_64-dev.tgz](https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-osx-x86_64-dev.tgz) |
-|MacOS-arm64| [faster_tokenizer-osx-arm64-dev.tgz](https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-osx-arm64-dev.tgz) |
+|Linux-x64| [fast_tokenizer-linux-x64-dev.tgz](https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-linux-x64-dev.tgz) |
+|Linux-aarch64| [fast_tokenizer-linux-aarch64-dev.tgz](https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-linux-aarch64-dev.tgz) |
+|Windows| [fast_tokenizer-win-x64-dev.zip](https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-win-x64-dev.zip) |
+|MacOS-x64| [fast_tokenizer-osx-x86_64-dev.tgz](https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-osx-x86_64-dev.tgz) |
+|MacOS-arm64| [fast_tokenizer-osx-arm64-dev.tgz](https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-osx-arm64-dev.tgz) |
### 环境依赖
@@ -35,15 +35,15 @@
## 2. 快速开始
-以下以Linux平台为例, 介绍如何使用FasterTokenizer CPP预编译包完成demo示例编译及运行。
+以下以Linux平台为例, 介绍如何使用FastTokenizer CPP预编译包完成demo示例编译及运行。
### 2.1 下载解压
```shell
-wget -c https://bj.bcebos.com/paddlenlp/faster_tokenizer/faster_tokenizer-linux-x64-dev.tgz
+wget -c https://bj.bcebos.com/paddlenlp/fast_tokenizer/fast_tokenizer-linux-x64-dev.tgz
-tar xvfz faster_tokenizer-linux-x64-dev.tgz
-# 解压后为faster_tokenizer目录
+tar xvfz fast_tokenizer-linux-x64-dev.tgz
+# 解压后为fast_tokenizer目录
```
### 2.1 编译
@@ -53,8 +53,8 @@ tar xvfz faster_tokenizer-linux-x64-dev.tgz
mkdir build
cd build
-# 运行cmake,通过指定faster_tokenizer包的路径,构建Makefile
-cmake .. -DFASTER_TOKENIZER_INSTALL_DIR=/path/to/faster_tokenizer
+# 运行cmake,通过指定fast_tokenizer包的路径,构建Makefile
+cmake .. -DFAST_TOKENIZER_INSTALL_DIR=/path/to/fast_tokenizer
# 编译
make
@@ -63,7 +63,7 @@ make
### 2.2 运行
```shell
-./ernie_faster_tokenizer_demo
+./ernie_fast_tokenizer_demo
```
diff --git a/fast_tokenizer/fast_tokenizer/demo/clip_fast_tokenizer_demo.cc b/fast_tokenizer/fast_tokenizer/demo/clip_fast_tokenizer_demo.cc
new file mode 100644
index 000000000000..064d9bd54c78
--- /dev/null
+++ b/fast_tokenizer/fast_tokenizer/demo/clip_fast_tokenizer_demo.cc
@@ -0,0 +1,66 @@
+/* Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License. */
+
+#include "fast_tokenizer/tokenizers/clip_fast_tokenizer.h"
+#include
+#include
+using namespace paddlenlp;
+
+template
+std::ostream& operator<<(std::ostream& os, const std::vector vec) {
+ os << "[";
+ for (int i = 0; i < vec.size(); ++i) {
+ if (i == 0) {
+ os << vec[i];
+ } else {
+ os << ", " << vec[i];
+ }
+ }
+ os << "]";
+ return os;
+}
+
+fast_tokenizer::tokenizers_impl::ClipFastTokenizer CreateClipFastTokenizer(
+ const std::string& vocab_path,
+ const std::string& merge_path,
+ uint32_t max_length,
+ bool pad = true) {
+ fast_tokenizer::tokenizers_impl::ClipFastTokenizer tokenizer(
+ vocab_path, merge_path, max_length);
+ if (pad) {
+ tokenizer.EnablePadMethod(fast_tokenizer::core::RIGHT,
+ tokenizer.GetPadTokenId(),
+ 0,
+ tokenizer.GetPadToken(),
+ &max_length,
+ nullptr);
+ }
+ return tokenizer;
+}
+
+int main() {
+ // 1. Define a clip fast tokenizer
+ auto tokenizer =
+ CreateClipFastTokenizer("clip_vocab.json", "clip_merges.txt", 77, true);
+ // 2. Tokenize the input strings
+ std::vector encodings;
+ std::vector texts = {
+ "a photo of an astronaut riding a horse on mars"};
+ tokenizer.EncodeBatchStrings(texts, &encodings);
+ for (auto&& encoding : encodings) {
+ auto ids = encoding.GetIds();
+ std::cout << ids << std::endl;
+ }
+ return 0;
+}
\ No newline at end of file
diff --git a/faster_tokenizer/faster_tokenizer/demo/ernie_faster_tokenizer_demo.cc b/fast_tokenizer/fast_tokenizer/demo/ernie_fast_tokenizer_demo.cc
similarity index 59%
rename from faster_tokenizer/faster_tokenizer/demo/ernie_faster_tokenizer_demo.cc
rename to fast_tokenizer/fast_tokenizer/demo/ernie_fast_tokenizer_demo.cc
index d97d509a04ae..6cf9812735c2 100644
--- a/faster_tokenizer/faster_tokenizer/demo/ernie_faster_tokenizer_demo.cc
+++ b/fast_tokenizer/fast_tokenizer/demo/ernie_fast_tokenizer_demo.cc
@@ -12,21 +12,20 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
-#include "faster_tokenizer/tokenizers/ernie_faster_tokenizer.h"
+#include "fast_tokenizer/tokenizers/ernie_fast_tokenizer.h"
#include
#include
using namespace paddlenlp;
int main() {
- using faster_tokenizer::core::EncodeInput;
- // 1. Define a ernie faster tokenizer
- faster_tokenizer::tokenizers_impl::ErnieFasterTokenizer tokenizer(
+ // 1. Define a ernie fast tokenizer
+ fast_tokenizer::tokenizers_impl::ErnieFastTokenizer tokenizer(
"ernie_vocab.txt");
// 2. Tokenize the input strings
// case 1: tokenize a single string
std::cout << "case 1: Tokenize a single string" << std::endl;
- faster_tokenizer::core::Encoding encoding;
- EncodeInput single_string =
+ fast_tokenizer::core::Encoding encoding;
+ std::string single_string =
"商赢环球股份有限公司关于延期回复上海证券交易所对"
"公司2017年年度报告的事后审核问询函的公告";
tokenizer.EncodePairStrings(single_string, &encoding);
@@ -34,16 +33,16 @@ int main() {
// case 2: tokenize a pair of strings
std::cout << "case 2: Tokenize a pair of strings" << std::endl;
- EncodeInput pair_string =
- std::pair{"蚂蚁借呗等额还款可以换成先息后本吗",
- "借呗有先息到期还本吗"};
- tokenizer.EncodePairStrings(pair_string, &encoding);
+ std::string text = "蚂蚁借呗等额还款可以换成先息后本吗";
+ std::string text_pair = "借呗有先息到期还本吗";
+
+ tokenizer.EncodePairStrings(text, text_pair, &encoding);
std::cout << encoding.DebugString() << std::endl;
// case 3: Tokenize a batch of single strings
std::cout << "case 3: Tokenize a batch of single strings" << std::endl;
- std::vector encodings;
- std::vector strings_list = {
+ std::vector encodings;
+ std::vector strings_list = {
"通过中介公司买了二手房,首付都付了,现在卖家不想卖了。怎么处理?",
"凌云研发的国产两轮电动车怎么样,有什么惊喜?",
"一辆车的寿命到底多长,最多可以开多久?"};
@@ -54,14 +53,16 @@ int main() {
// case 4: Tokenize a batch of pair strings
std::cout << "case 4: Tokenize a batch of pair strings" << std::endl;
- std::vector pair_strings_list = {
- std::pair({"花呗自动从余额宝扣款,需要我自己设置吗", "支付宝余额会自动还花呗吗"}),
- std::pair({"这个蚂蚁花呗能恢复正常用不", "我的蚂蚁花呗 怎么用不了"}),
- std::pair({"在经济的另一次转变中,人们发现在低地农场饲养羔羊更具成本效益,部分原因"
- "是那里有更丰富、更有营养的牧场,因此湖地农场的利润变得更少。",
- "人们发现,经济的另一个转变更有营养。"}),
- };
- tokenizer.EncodeBatchStrings(pair_strings_list, &encodings);
+ std::vector texts = {
+ "花呗自动从余额宝扣款,需要我自己设置吗",
+ "这个蚂蚁花呗能恢复正常用不",
+ "在经济的另一次转变中,人们发现在低地农场饲养羔羊更具成本效益,部分原因"
+ "是那里有更丰富、更有营养的牧场,因此湖地农场的利润变得更少。"};
+ std::vector text_pairs = {
+ "支付宝余额会自动还花呗吗",
+ "我的蚂蚁花呗 怎么用不了",
+ "人们发现,经济的另一个转变更有营养。"};
+ tokenizer.EncodeBatchStrings(texts, text_pairs, &encodings);
for (auto&& encoding : encodings) {
std::cout << encoding.DebugString() << std::endl;
}
diff --git a/faster_tokenizer/faster_tokenizer/models/CMakeLists.txt b/fast_tokenizer/fast_tokenizer/models/CMakeLists.txt
similarity index 56%
rename from faster_tokenizer/faster_tokenizer/models/CMakeLists.txt
rename to fast_tokenizer/fast_tokenizer/models/CMakeLists.txt
index 05c568cb9a87..f51706a77035 100644
--- a/faster_tokenizer/faster_tokenizer/models/CMakeLists.txt
+++ b/fast_tokenizer/fast_tokenizer/models/CMakeLists.txt
@@ -1,3 +1,3 @@
cc_library(models
- SRCS wordpiece.cc faster_wordpiece.cc bpe.cc unigram.cc
+ SRCS wordpiece.cc fast_wordpiece.cc bpe.cc unigram.cc
DEPS core json trie failure icuuc icudata lattice utils)
diff --git a/faster_tokenizer/faster_tokenizer/models/bpe.cc b/fast_tokenizer/fast_tokenizer/models/bpe.cc
similarity index 98%
rename from faster_tokenizer/faster_tokenizer/models/bpe.cc
rename to fast_tokenizer/fast_tokenizer/models/bpe.cc
index 89e3a7464779..ad80d2752d6f 100644
--- a/faster_tokenizer/faster_tokenizer/models/bpe.cc
+++ b/fast_tokenizer/fast_tokenizer/models/bpe.cc
@@ -19,12 +19,12 @@ limitations under the License. */
#include