New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

微调能否调整参数，使微调脚本可以在GPU T4或者CPU上进行微调 #125

Open

dsscclg opened this issue Mar 8, 2024 · 1 comment

dsscclg commented Mar 8, 2024

RuntimeError: FlashAttention only supports Ampere GPUs or newer.

微调能否调整参数，使微调脚本pretrain_yuan2.0_2.1B_sft.sh可以在GPU T4或者CPU上进行微调？

Collaborator

zhaoxudong01 commented Mar 8, 2024

目前使用pretrain_yuan2.0_2.1B_sft.sh不支持CPU微调，T4可以删除use_flash_attn选项，注释调用flash attn的位置试一下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment