The problem with visualization during inference

[wujiang0156]

Your algorithm proposed in the paper is a fantastic idea. However, I have a few questions to ask you. When I use the ChangeCLIP_best_weights you provided to visualize the inference results of images in the test directory of datasets like LEVIR, I found the results to be unsatisfactory. Why are my visualization results so poor? Looking forward to your reply.
GT:

                                    predict

This is my code：
parser.add_argument('--file_list', default='D:/pythonwork/Data/LEVIR-CD/LEVIR-CD256256CLIP/test.txt', help='Image file')
parser.add_argument('--config', default='configs/0cd_ce/changeclip_levir.py')
parser.add_argument('--checkpoint', default='checkpoint/best_mIoU_iter_17000.pth')

Could you please share the visualization process and code you used? Thank you.

[dyzy41]

I guess the size of your input image is wrong. In fact, the input size should be 256x256. The inference method I designed in the paper is as follows:
(1) First, divide the large image of the test set into 256x256 according to the splitting method of the training set.
(2) Perform inference on slices and obtain inference results
(3) The inference results are spliced and integrated into a large picture, and the accuracy obtained is the accuracy mentioned in the article.
In addition, in the config file of the code, there is configs/0cd_ce/changeclip_levir_test.py, line71, test_cfg=dict(mode='slide', crop_size=(256, 256), stride=(128, 128)). If you input a large image directly and follow the sliding window prediction provided by mmseg, you will get better accuracy indicators.

[wujiang0156]

Thank you for your reply. In fact, I divided the large images in the test set into small blocks of 256x256 and performed inference on these slices. If the inference results of the slices are not good, then stitching them into a large image will certainly not be good either. However, I did not use overlapping slices for inference. Could this be the problem? I will try to perform inference with overlapping slices. @dyzy41

[dyzy41]

You can try this config.py for test, configs/0cd_ce/changeclip_levir_test.py

[wujiang0156]

@dyzy41
Thank you for your reply.
I used changeclip_levir_test.py for testing, and indeed, the results were better than my previous ones.

Additionally, I have an annother question: Did you use the same number of training epochs for your comparative experiments? For example, did you use 40k epochs for your method and also 40k epochs for Changerformer? Or did you follow the number of epochs set by each paper for comparison? The number of training epochs can affect accuracy, and if the training epochs are different for comparison, there may be differences in the results. How did you set the epochs for other methods?
Looking forward to your reply, thank you very much.

[dyzy41]

"When training a model, if you find that the loss has converged, you can stop the training. I trained the model in this way."

[COUJIALUO]

"Hello, I'd like to know how you managed to get the prediction code to work, as I've set the parameters the same as yours, but I encountered an error during prediction."
Traceback (most recent call last):
File "/root/autodl-tmp/ChangeCLIP-release/tools/infer_folderCD.py", line 72, in
main()
File "/root/autodl-tmp/ChangeCLIP-release/tools/infer_folderCD.py", line 64, in main
results = model.test_step(data)
File "/root/miniconda3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
return self._run_forward(data, mode='predict') # type: ignore
File "/root/miniconda3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward
results = self(**data, mode=mode)
File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/base.py", line 96, in forward
return self.predict(inputs, data_samples)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/ChangeCLIPCD.py", line 514, in predict
seg_logits = self.inference(inputs, batch_img_metas)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/ChangeCLIPCD.py", line 747, in inference
seg_logit = self.whole_inference(inputs, batch_img_metas)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/ChangeCLIPCD.py", line 718, in whole_inference
seg_logits = self.encode_decode(inputs, batch_img_metas)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/ChangeCLIPCD.py", line 219, in encode_decode
textA, textB = self.get_cls_text(batch_img_metas, False)
File "/root/autodl-tmp/ChangeCLIP-release/mmseg/models/segmentors/ChangeCLIPCD.py", line 388, in get_cls_text
foreA = ', '.join(['remote sensing image foreground objects']+img_infos[i]['jsonA'])
KeyError: 'jsonA'

[COUJIALUO]

@wujiang0156

[wujiang0156]

@COUJIALUO
Just predict according to the method provided by the authors above.