Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

other public datasets,overfitting issue #135

Open
YHaooo-4508 opened this issue Aug 2, 2023 · 6 comments
Open

other public datasets,overfitting issue #135

YHaooo-4508 opened this issue Aug 2, 2023 · 6 comments

Comments

@YHaooo-4508
Copy link

Hello, I am trying to adapt the MPII_3DHP dataset on your code, but it's totally overfitting
First I train it on H36M+MPII,the MPJPE is about 50+. This is normal
Then i change the dataset using MPII_3DHP+MPII to train. 50MPJPE in trainset but 200+ in validate dataset.
Have you conducted experiments on MPII_3DHP dataset? Why is there such a serious overfitting?

@mks0601
Copy link
Owner

mks0601 commented Aug 2, 2023

I'm not clear about your experiments.

  1. First I train it on H36M+MPII,the MPJPE is about 50+. This is normal -> what is the test set?
  2. Then i change the dataset using MPII_3DHP+MPII to train. 50MPJPE in trainset but 200+ in validate dataset. -> what are the training and validation sets?

@YHaooo-4508
Copy link
Author

I'm not clear about your experiments.

  1. First I train it on H36M+MPII,the MPJPE is about 50+. This is normal -> what is the test set?
  2. Then i change the dataset using MPII_3DHP+MPII to train. 50MPJPE in trainset but 200+ in validate dataset. -> what are the training and validation sets?

1、train with H36M + MPII2D train dataset;test with H36M validate dataset
2、train with MPII_3DHP+MPII2D train dataset;test with MPII_3DHP validate dataset

@mks0601
Copy link
Owner

mks0601 commented Aug 3, 2023

I don't think this is from overfitting. I guess there is something wrong in your evaluation code?

@YHaooo-4508
Copy link
Author

I don't think this is from overfitting. I guess there is something wrong in your evaluation code?

I checked the code and found that there was an issue with bbox, which has been fixed. However, the results of subsequent testing were still overfitting. The validation set MPJPE was around 150, trainset is about 60. Here are some of my visualization results. Some samples also have poor results
image
image
image
image

@mks0601
Copy link
Owner

mks0601 commented Aug 4, 2023

Error 60 on training set means definitely underfitting

@YHaooo-4508
Copy link
Author

Error 60 on training set means definitely underfitting

Firstly, thank you for communicating with me.
I know that continuing training may improve performance in the training set, but at this point, the validation set error no longer decreases, so I stopped training, and there was a significant difference between the validation set error and the training set error.
By the way, during my training and validation process, I did not use object detection algorithms to predict bbox, but instead used key point positions to fit the bounding box. I am not sure if this method will affect the training.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants