-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Questions about input of VLDM #16
Comments
Hi!
|
Thanks for your prompt reply! Maybe I git it wrong. The objective of this paper is given 2 views, generate new views directly. So at the trainning stage, all we got is input_rgb(context view), and the batch_rgb(query view) is what we want to generate. How can it be used as input to vae encoder and diffusion model? Or this framework is not instance-specific. We train the model on a lot of instances, and generalize to new instance at distillation stage? If so, can you explain a little bit more about it? Thank you so much for your help! |
Sorry I got another question, have you tried single view reconstruction on this framework? Thanks for your help! |
VLDM is not instance specific. It is trained on a lot of instances. During training, the model sees input_rgb, input_cameras, and query_cameras. We have ground truth for query_rgb, which we supervise the model with. Since we use a latent diffusion model, the model predicts latent codes internally. Single view reconstruction requires a bit more careful consideration since the object scale in ambiguous. |
Usually cublas_status_not_initialized means the program or environment cannot find a cuda gpu. |
Excellent job!
![image](https://private-user-images.githubusercontent.com/53629747/275556675-2fa76143-866d-4fea-8151-50ded7a925b3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ2NzY2NDksIm5iZiI6MTcyNDY3NjM0OSwicGF0aCI6Ii81MzYyOTc0Ny8yNzU1NTY2NzUtMmZhNzYxNDMtODY2ZC00ZmVhLTgxNTEtNTBkZWQ3YTkyNWIzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI2VDEyNDU0OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRjMzBjMjc5NTA3MTQ5NzNmOTIwNmU3NjFjNWRiYzdkZGY1ZDRjZmY1NjQzMDc5YWQ5YTM2ZDkxMDQ2NmEwYTQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.trxY2ZGe3bNr3hTrBCGUDDu-RBQ-8LlYDAT7D2uEArs)
I've got some questions about the code.
The input of VLDM is from batch_rgb(aka query_rgb)? But we didn't have the query_rgb at the training stage?
And where the z_scale_factor from? why it was fixed to be 0,18215?
Thanks!
The text was updated successfully, but these errors were encountered: