Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the scripts execute order #8

Open
Veluriyam opened this issue Sep 27, 2024 · 1 comment
Open

About the scripts execute order #8

Veluriyam opened this issue Sep 27, 2024 · 1 comment

Comments

@Veluriyam
Copy link

bash scripts/train_llama.sh
[2024-09-27 16:58:30,082] [estimate.py:67] pretrained_model_path: /root/yp/LLM-Safeguard-main/code/meta-llama/Llama-2-7b-chat-hf
[2024-09-27 16:58:30,082] [estimate.py:67] config: sampling
[2024-09-27 16:58:30,082] [estimate.py:67] output_path: ./estimations
[2024-09-27 16:58:30,082] [estimate.py:67] system_prompt_type: all
[2024-09-27 16:58:30,082] [estimate.py:67] n_splits: 10
[2024-09-27 16:58:30,089] [utils.py:94] GPU 0: 0.27 GB / 32.00 GB
[2024-09-27 16:58:30,095] [utils.py:94] GPU 1: 15.65 GB / 32.00 GB
[2024-09-27 16:58:30,224] [estimate.py:89] /root/yp/LLM-Safeguard-main/code/meta-llama/Llama-2-7b-chat-hf
[2024-09-27 16:58:30,226] [estimate.py:98] Running harmful
[2024-09-27 16:58:30,694] [estimate.py:124] Running harmless
[2024-09-27 16:58:30,922] [utils.py:46] File eval_results/sampling/Llama-2-7b-chat-hf_custom.csv does not exist, exiting
[2024-09-27 16:58:34,948] [train.py:107] pretrained_model_path: /root/yp/LLM-Safeguard-main/code/meta-llama/Llama-2-7b-chat-hf
[2024-09-27 16:58:34,948] [train.py:107] config: sampling
[2024-09-27 16:58:34,948] [train.py:107] system_prompt_type: all
[2024-09-27 16:58:34,948] [train.py:107] prompt_length: default
[2024-09-27 16:58:34,948] [train.py:107] output_path: ./trained_prompts
[2024-09-27 16:58:34,948] [train.py:107] ablate_norm: False
[2024-09-27 16:58:34,948] [train.py:107] ablate_refu: False
[2024-09-27 16:58:34,948] [train.py:107] ablate_harm: False
Hello, fllow your Readme, after I bash scripts/forward.sh with forward_harmless.sh, when I bash scripts/train_llama.sh it makes the error that can't find eval_results/sampling/Llama-2-7b-chat-hf_custom.csv, and I noted that evaluate.py generates the . /eval_results file in order to get the scores in utils.py . So is there something wrong with my execution order, do I need to execute forward.sh and forward_less.sh first, then execute evaluate.py to get the scores so that I can execute estimate.py to generate the csv file, and if so why is there no such step in train_meta.sh?
你好,看了你的 Readme,在我执行 scripts/forward.sh和forward_harmless.sh 之后,当我执行 scripts/train_llama.sh 时,报错 eval_results/sampling/Llama-2-7b-chat-hf_custom.csv ,我注意到 evaluate.py 会生成 ./eval_results 文件,以便在 utils.py 中获取分数。那么我的执行顺序是否有问题,我是否需要先执行 forward.sh 和 forward_less.sh,然后执行 evaluate.py 获得分数,这样才能执行 estimate.py 生成 csv 文件,如果是这样,为什么 train_meta.sh 中没有这样的步骤?

@Huahuoy
Copy link

Huahuoy commented Dec 9, 2024

但是一开始outputs文件夹是空的啊,执行运行会报错logging.info(f"File {res_file} does not exist, skip")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants