How to evaluate if I get a checkpoint file?

[JiuqingDong]

Hi, could you tell me how can I evaluate the model when I finished the tuning?

[JiuqingDong]

Oh, I already solved this issue and forgot to put the solution here.

Step 1: Save the full model for best_metric.
if curr_acc >= best_metric:
model_state = self.model.state_dict()
out_path = os.path.join(self.cfg.OUTPUT_DIR, "best_model.pth")
torch.save(model_state, out_path)

Step 2: Train(Tuning) the model and get the checkpoint file. Besides,
Use 'model.load_state_dict(torch.load(cfg.MODEL.WEIGHT_PATH))' to load the checkpoint.

Step 3: Create a shell file like this:

python tune_fgvc.py
--train-type "prompt"
--config-file configs/prompt/cub.yaml
MODEL.TYPE "vit"
DATA.BATCH_SIZE "128"
MODEL.PROMPT.DEEP "True"
MODEL.PROMPT.DROPOUT "0.1"
MODEL.PROMPT.NUM_TOKENS "10"
DATA.FEATURE "sup_vitb16_imagenet21k"
MODEL.MODEL_ROOT "models/"
OUTPUT_DIR "./prompt_tuning/CUB/"
MODEL.WEIGHT_PATH "/Path/val_CUB_model.pth"
SOLVER.TOTAL_EPOCH '0' \

By the way, It is a naive way to do evaluation. I want to know how to use logits.pth to do the evaluation, because in your code, you just save the {"targets": total_targets, "joint_logits": joint_logits} in checkpoint file.