intergate robustsam to hf more easily

[jadechoghari]

Hello Jade's here,
RobustSAM sounds very interesting—congratulations on the paper!
I wonder if you would consider adding RobustSAM to the Hugging Face Hub. Doing so increase the model's visibility and make it easier for everyone to use.
Right now, only the checkpoints are uploaded: https://huggingface.co/robustsam/robustsam/tree/main).
It can be easily compatible with the Hugging Face Transformers library.

Can work on that!

Best.

[robustsam]

Thanks for your advice. We are now working on this!

[jadechoghari]

Hey! I just completed the HF implementation of RobustSAM 🎉 (in case your model card is not done :) )

Easily accessible and usable for everyone. You can even track the download stats. 🤗

Please let me know if you need any update in the README.

We will have three model cards:
https://huggingface.co/jadechoghari/robustsam-vit-base
https://huggingface.co/jadechoghari/robustsam-vit-large
https://huggingface.co/jadechoghari/robustsam-vit-huge

I also think it would be cool to provide a Gradio demo with some examples of degraded images and more, so non-technical people can interact with the model!
Let me know if you want a gradio demo!

Best.

[jadechoghari]

will ping @robustsam @weitingchen83 :) !

[robustsam]

@jadechoghari Thanks for your help. We will add your version to our project page for other users.

[jadechoghari]

yes great, we'll also integrate gradio :), with some few image samples.
I'll keep you posted!

[jadechoghari]

@robustsam Regarding the Gradio demo, we can implement a very cool demo (HF Space) inspired by https://huggingface.co/spaces/merve/OWLSAM
We can add the Gradio file to this repo locally & online along with your selected demo images!
The demo will allow people to easily test RobustSAM with text prompts as well.
You can check the work we did on VFusion3D (by Meta) here: VFusion3D GitHub Repo.

I'll open a PR for it if you want and ping the Gradio maintainer: @yvrjsharma for context: RobustSAM is an interesting model developed by Snap Research.

Best 🚀

[yvrjsharma]

Thanks for the tag @jadechoghari. RobustSAM looks great! It'd be awesome to have a Gradio demo locally as well as on Spaces. Feel free to open a Community Grant Request for A100s (ZeroGPU) and we'll be happy to help with it.
We can also help in amplifying the project on SM through posts and Twitter broadcasts @weitingchen83 if that will be helpful.

[robustsam]

Hi @jadechoghari ,

We are willing to proceed. If you have any questions, please feel free to contact me. Thank you.

[jadechoghari]

Great! I'll open a PR soon.

Everything is done on my end. The Gradio demo works nicely locally (in Jupyter, Colab, etc.), but when deployed on Spaces, the image does not display.

We are using gr.AnnotatedImages (Gradio version: 4.40.0). You can find the code here: app.py.

Any thoughts on why this might be happening, @yvrjsharma?

Thanks!

[jadechoghari]

I think everything is done with this issue. We’ve added Gradio and HF model cards!

Before closing, @yvrjsharma, could we feature the space on Gradio's social media/Twitter to amplify the model/project?

We’re using ZeroGPU for RobustSAM here 🥳: ZeroGPU: https://huggingface.co/spaces/jadechoghari/robustsam

Thanks again 🚀