Support Distill Depth Anything

[oxysoft]

Model description

Excellent new depth estimation model just dropped. Would be great to be able to use it from the hub in a few lines like the other models.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

https://github.com/Westlake-AGI-Lab/Distill-Any-Depth

[qubvel]

Hi @oxysoft! Thanks for opening the feature request. 🤗 As far as I understand adding support for this model should be straightforward because the model has the same architecture as DepthAnything (DA), so the ideas is to use DA conversion script to convert checkpoints and upload them to the Hub in the new format, compatible with 🤗 Transformers.

[keetrap]

Hi @qubvel ,

I’m interested in working on this, but I have a few questions:

DA Conversion Script: From what I understand, the current DA conversion script won't work as-is because the weights are already in safetensors format, and the key names are also different. So, I assume we’ll need to modify the DA script to handle this format and key name differences. Would you agree?

Uploading the Files: Once the conversion to the compatible format is complete, where should the converted files be uploaded?
Also, I’ve created a compatible version of the files, which you can find here: Distil-Any-Depth-Large-hf

Model Directory: Regarding the location of the changes, should they go into the existing "depth_anything" model directory, or would it be better to create a new model dir (e.g., something like "distill_depth_anything") to house these changes?

Sorry if my questions sound basic—I'm new to contributing models to the Transformers library and just want to make sure I’m following the correct process!

Thanks again for your help!

[keetrap]

@qubvel, just a gentle reminder.

[qubvel]

Hey @keetrap, thanks for the ping, I missed the message 🤗

DA Conversion Script: From what I understand, the current DA conversion script won't work as-is because the weights are already in safetensors format, and the key names are also different. So, I assume we’ll need to modify the DA script to handle this format and key name differences. Would you agree?

Yes, in case the current script is not working, we can have a second version of the script which convert distil model

Uploading the Files: Once the conversion to the compatible format is complete, where should the converted files be uploaded? Also, I’ve created a compatible version of the files, which you can find here: Distil-Any-Depth-Large-hf

Once it compele, we can tag model authors here to ask if they want to manage it under their own org on HF Hub, otherwise we can create a community org and put it there.

Model Directory: Regarding the location of the changes, should they go into the existing "depth_anything" model directory, or would it be better to create a new model dir (e.g., something like "distill_depth_anything") to house these changes?

Can you provide a few more details on the changes? If it's super minor, we can have this integrated under the same model, otherwise, it's better to submit a new PR for a new model. You can open a PR and tag me and we will decide whether we should create a new model for that

Thanks for interest in contribution!