This is the implementation of COLING 2022 paper:
Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning. [arXiv]
Python == 3.7
- Clone repository
$ git clone --recursive [email protected]:nu-dialogue/antor.git $ cd antor
- Install ConvLab-2
cd ConvLab-2 pip install -e . cd ../
- Install antor
pip install -e . python -m spacy download en_core_web_sm
It is ok to ignore pip's dependency error with ConvLab-2.
You can reproduce the three experiments performed in the paper by following these steps.
Before running the experiment, the NLG and NLU models must be prepared respectively.
- GPT-2 (Base model for reinforcement learning) and SC-GPT
- Go to
sys_nlg/gpt2
andsys_nlg/scgpt
and prepare each model following theREADME.md
in the directory, respectively.
- Go to
- SC-LSTM
- (Since we use pre-trained models available in the ConvLab-2, there are no steps required.)
- MILU and BERT NLU
- Go to
user_nlu/milu
anduser_nlu/bert
and prepare the each model following theREADME.md
in the directory, respectively.
- Go to
- Simply fine-tune the NLG (GPT-2) with NLU (BERT NLU or MILU)
- Go to
experiments/ppo
and follow theREADME.md
in the directory.
- Go to
- Evaluate the fine-tuned model
- Go to
experiments/evaluate_model
and follow theREADME.md
in the directory.
- Go to
- Build the confusion matrix for ASR error simulation
- Build the confusion matrix by performing the steps described in each
README.md
inexperiments/text2speech_data
,experiments/noisy_speech_data
,experiments/speech2text_data
, andexperiments/speech_error_simulation
, in that order.
You can skip this step by directly using the final confusion matrices we used in our paper. The usage can be found in
experiments/speech_error_simulation
. In addition, we also publish the noisy transcribed text data used to build the confusion matrices. Seeexperiments/speech2text_data
. - Build the confusion matrix by performing the steps described in each
- Fine-tune NLG in ASR error simulation
- Go to
experiments/ppo
and follow theREADME.md
in the directory.
- Go to
- Evaluate the fine-tuned model
- Go to
experiments/evaluate_model
and follow theREADME.md
in the directory.
- Go to
- Prepare vocabulary level
- Go to
experiments/vocabulary_level
and follow theREADME.md
in the directory.
- Go to
- Train NLU models with only certain vocabulary levels of data
- Go to
user_nlu/milu
oruser_nlu/bert
and train each model following theREADME.md
in the directory.
- Go to
- Fine-tune NLG
- Go to
experiments/ppo
and follow theREADME.md
in the directory.
- Go to
- Evaluate the fine-tuned model
- Go to
experiments/evaluate_model
and follow theREADME.md
in the directory.
- Go to
@article{ohashi2022adaptive,
title={Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning},
author={Ohashi, Atsumoto and Higashinaka, Ryuichiro},
journal={arXiv preprint arXiv:2209.07873},
year={2022}
}