You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to reproduce the results of a paper. Currently, I am using the 'chatgpt-detector-roberta' model and tokenizer directly imported from Hugging Face. According to the information on their website, this model was trained on a mixed dataset. In the paper, the F1 score for 'raw-full' should be 99.44. However, I am unable to achieve this result, The dataset used was downloaded from the Google Drive link provided in the README of hc3, and here are the results I obtained.
{'0': {'precision': 0.9994103425909546, 'recall': 0.9951852504256943, 'f1-score': 0.9972933215651661, 'support': 17031.0}, '1': {'precision': 0.9898640296662546, 'recall': 0.9987528061860813, 'f1-score': 0.994288552272163, 'support': 8018.0}, 'accuracy': 0.9963271986905665, 'macro avg': {'precision': 0.9946371861286046, 'recall': 0.9969690283058878, 'f1-score': 0.9957909369186646, 'support': 25049.0}, 'weighted avg': {'precision': 0.9963546382901745, 'recall': 0.9963271986905665, 'f1-score': 0.9963315170942771, 'support': 25049.0}}
The text was updated successfully, but these errors were encountered:
我尝试复现中论文中的结果,现在我是直接导入hugging face上的chatgpt-detector-roberta作为model和tokenizer,根据页面上的描述这是由mixed数据集训练的,在论文中对raw-full的F1 score应该为99.44,但我没办法得到这个数据,我使用的数据集是在hc3中readme中的谷歌网盘下载的,以下是我得到的结果
{'0': {'precision': 0.9994103425909546, 'recall': 0.9951852504256943, 'f1-score': 0.9972933215651661, 'support': 17031.0}, '1': {'precision': 0.9898640296662546, 'recall': 0.9987528061860813, 'f1-score': 0.994288552272163, 'support': 8018.0}, 'accuracy': 0.9963271986905665, 'macro avg': {'precision': 0.9946371861286046, 'recall': 0.9969690283058878, 'f1-score': 0.9957909369186646, 'support': 25049.0}, 'weighted avg': {'precision': 0.9963546382901745, 'recall': 0.9963271986905665, 'f1-score': 0.9963315170942771, 'support': 25049.0}}
I am trying to reproduce the results of a paper. Currently, I am using the 'chatgpt-detector-roberta' model and tokenizer directly imported from Hugging Face. According to the information on their website, this model was trained on a mixed dataset. In the paper, the F1 score for 'raw-full' should be 99.44. However, I am unable to achieve this result, The dataset used was downloaded from the Google Drive link provided in the README of hc3, and here are the results I obtained.
{'0': {'precision': 0.9994103425909546, 'recall': 0.9951852504256943, 'f1-score': 0.9972933215651661, 'support': 17031.0}, '1': {'precision': 0.9898640296662546, 'recall': 0.9987528061860813, 'f1-score': 0.994288552272163, 'support': 8018.0}, 'accuracy': 0.9963271986905665, 'macro avg': {'precision': 0.9946371861286046, 'recall': 0.9969690283058878, 'f1-score': 0.9957909369186646, 'support': 25049.0}, 'weighted avg': {'precision': 0.9963546382901745, 'recall': 0.9963271986905665, 'f1-score': 0.9963315170942771, 'support': 25049.0}}
The text was updated successfully, but these errors were encountered: