Improved meta-data for models #5170
Conversation
💊 CI failures summary and remediationsAs of commit 8cae313 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
| url="https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth", | ||
| transforms=CocoEval, | ||
| meta={ | ||
| "task": "image_object_detection", |
There was a problem hiding this comment.
I'm unclear on what to put here.
Indeed, this model could also be categorized as image_instance_segmentation. Maybe having task be a list would be of help here?
Ensuring that this is adapted with paperswithcode categorization would be good as well https://paperswithcode.com/paper/mask-r-cnn
There was a problem hiding this comment.
Yeah same here. I'll leave as is for now and review in more detail on the future.
Summary: * Improved meta-data for models. * Addressing comments from code-review. * Add parameter count. * Fix linter. Reviewed By: sallysyw Differential Revision: D33479281 fbshipit-source-id: 7a133324ed5a289a0ac89522b0d4a38ce8b201e0
|
|
||
|
|
||
| _COMMON_META = { | ||
| "task": "image_object_detection", |
There was a problem hiding this comment.
image keypoint detection. I guess?
|
Could we add size of the model in MBs #4996 ? Anything about FLOPS of computation / Runtime statistics too would be cool. (Although it's hard to say as runtime highly various from system to system and configs) |
|
@oke-aditya Sounds like good suggestions. Would you like to send a PR that adds model sizes in? Ideally with a unit-test to compare the declared meta-data size with the actual size (you are guaranteed that they are available locally but you might need to be creative on how to do this; I got some similar checks here in this PR for model size in params). FLOPS would be really cool, but it's not always straightforward to automatically estimate this for all models (for example detection models). I'm open for any ideas. BTW we've got a feedback issue at #5088 were we keep proposals about the new API centralized. Feel free to post more there if you have additional ideas. :) |
|
I would need some time for this as it's been hectic week at office :( (It's literally odd to code in Java in morning and python at night) |
4 participants
Enhance the meta-data information stored for each model by including the following info:
cc @datumbox @bjuncek