Skip to content

✨ Better typing for LIBRARY_TASK_MAPPING_EXCLUDING_TRANSFORMERS #552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SBrandeis merged 5 commits into from
Dec 14, 2022
Merged

✨ Better typing for LIBRARY_TASK_MAPPING_EXCLUDING_TRANSFORMERS #552

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
02adbd8
✨ Better typing for LIBRARY_TASK_MAPPING_EXCLUDING_TRANSFORMERS
SBrandeis Dec 12, 2022
d156f2d
Proposal @SBrandeis
julien-c Dec 12, 2022
7548bbc
haha search'n'replace gone wrong
julien-c Dec 12, 2022
64e8088
Proposed typing fix
julien-c Dec 14, 2022
df779f3
Update js/src/lib/interfaces/Libraries.ts
SBrandeis Dec 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions js/src/lib/interfaces/Libraries.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ export enum ModelLibrary {
"pythae" = "Pythae",
}

export const ALL_MODEL_LIBRARY_KEYS = Object.keys(ModelLibrary) as (keyof typeof ModelLibrary)[];
export type ModelLibraryKey = ModelLibraryKey;
export const ALL_MODEL_LIBRARY_KEYS = Object.keys(ModelLibrary) as ModelLibraryKey[];


/**
Expand Down Expand Up @@ -390,7 +391,7 @@ model = AutoModel.load_from_hf_hub("${model.id}")`;



export const MODEL_LIBRARIES_UI_ELEMENTS: { [key in keyof typeof ModelLibrary]?: LibraryUiElement } = {
export const MODEL_LIBRARIES_UI_ELEMENTS: { [key in ModelLibraryKey]?: LibraryUiElement } = {
// ^^ TODO(remove the optional ? marker when Stanza snippet is available)
"adapter-transformers": {
btnLabel: "Adapter Transformers",
Expand Down
74 changes: 38 additions & 36 deletions js/src/lib/interfaces/Types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import type { ModelLibraryKey } from "./Libraries";

// Warning: order of modalities here determine how they are listed on the /tasks page
export const MODALITIES = [
"cv",
Expand Down Expand Up @@ -758,88 +760,88 @@ export interface TransformersInfo {
* This mapping is generated automatically by "python-api-export-tasks" action in huggingface/api-inference-community repo upon merge.
* Ref: https://github.com/huggingface/api-inference-community/pull/158
*/
export const LIBRARY_TASK_MAPPING_EXCLUDING_TRANSFORMERS: Record<string, Array<string>> = {
"adapter_transformers": [
export const LIBRARY_TASK_MAPPING_EXCLUDING_TRANSFORMERS: Partial<Record<ModelLibraryKey, PipelineType[]>> = {
"adapter-transformers": [
"question-answering",
"text-classification",
"token-classification"
"token-classification",
],
"allennlp": [
"question-answering"
"question-answering",
],
"asteroid": [
"audio-source-separation",
"audio-to-audio"
// "audio-source-separation",
Copy link
Member

@julien-c julien-c Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not a valid pipeline tag)

"audio-to-audio",
],
"diffusers": [
"text-to-image"
],
"doctr": [
"object-detection"
"text-to-image",
],
// "doctr": [
// "object-detection",
// ],
Copy link
Contributor Author

@SBrandeis SBrandeis Dec 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented because it's absent from ModelLibrary

Copy link
Contributor Author

@SBrandeis SBrandeis Dec 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for superb & k2_sherpa

Copy link
Member

@coyotte508 coyotte508 Dec 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be handled from the source huggingface/api-inference-community? (see huggingface/api-inference-community#158)

Copy link
Contributor

@xianbaoqian xianbaoqian Dec 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@osanseviero Any thoughts how to handle the discrepancy. Is the inference API working for doctr, superb & k2_sherpa? Or do we need to add these to ModelLibrary?

I didn't find anything related to k2_sherpa in the hub. I had a quick glimpse on a few superb models and they seem to be using transformer for inference API (?) Most doctr repos are dummy models and the inference API is unable to decide the pipeline_tag.

Copy link
Contributor

@osanseviero osanseviero Dec 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the inference API working for doctr, superb & k2_sherpa? Or do we need to add these to ModelLibrary?

  • Superb repos use the Community Inference API, but given they are not a library and their usage is more for a competition, they were not added to the Libraries.ts (confirmed with @lewtun). From a discussion, we think https://huggingface.co/models?library=superb&sort=modified are mostly old legacy models and it's ok to remove them from the Inference API, as the official ones now use transformers. (the old integration allowed users to specify their own requirements and inference code but that was deprecated)
  • I don't have context in the k2-sherpa integration, so maybe makes sense to remove entirely from the community API or exclude from the exporter
  • https://huggingface.co/models?library=doctr&sort=modified is just one model so I'm fine removing doctr

Copy link
Member

@julien-c julien-c Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for me this PR is ready to merge

Copy link
Contributor

@xianbaoqian xianbaoqian Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think adding the two definitely makes sense.

Another issue is that people have been using k2 in HuggingFace tags and k2_sherpa in api-inference-community. I think the easiest fix is to rename k2_sherpa to k2. I'll propose a PR in api-inference-community!

Copy link
Member

@julien-c julien-c Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(also see internal PR https://github.com/huggingface/moon-landing/commit/e9557f8c848abbb98e48e8c4023ac6526022273e)

Copy link
Member

@julien-c julien-c Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and yes renaming k2_sherpa to k2 sounds perfect

Copy link
Contributor

@osanseviero osanseviero Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

64e8088 was exactly what I had in mind when we discussed @xianbaoqian. :) Thanks for doing it @julien-c!

"espnet": [
"text-to-speech",
"automatic-speech-recognition"
"automatic-speech-recognition",
],
"fairseq": [
"text-to-speech",
"audio-to-audio"
"audio-to-audio",
],
"fastai": [
"image-classification"
"image-classification",
],
"fasttext": [
"feature-extraction",
"text-classification"
"text-classification",
],
"flair": [
"token-classification"
],
"k2_sherpa": [
"automatic-speech-recognition"
"token-classification",
],
// "k2_sherpa": [
// "automatic-speech-recognition",
// ],
"keras": [
"image-classification"
"image-classification",
],
"nemo": [
"automatic-speech-recognition"
"automatic-speech-recognition",
],
"paddlenlp": [
"conversational",
"fill-mask"
"fill-mask",
],
"pyannote_audio": [
"automatic-speech-recognition"
"pyannote-audio": [
"automatic-speech-recognition",
],
"sentence_transformers": [
"sentence-transformers": [
"feature-extraction",
"sentence-similarity"
"sentence-similarity",
],
"sklearn": [
"tabular-classification",
"tabular-regression",
"text-classification"
"text-classification",
],
"spacy": [
"token-classification",
"text-classification",
"sentence-similarity"
"sentence-similarity",
],
"speechbrain": [
"audio-classification",
"audio-to-audio",
"automatic-speech-recognition",
"text-to-speech",
"text2text-generation"
"text2text-generation",
],
"stanza": [
"token-classification"
],
"superb": [
"automatic-speech-recognition",
"speech-segmentation"
"token-classification",
],
// "superb": [
// "automatic-speech-recognition",
// "speech-segmentation",
// ],
"timm": [
"image-classification"
]
}
"image-classification",
],
};
4 changes: 2 additions & 2 deletions tasks/src/Types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { ModelLibrary } from "../../js/src/lib/interfaces/Libraries";
import type { ModelLibraryKey } from "../../js/src/lib/interfaces/Libraries";
import type { PipelineType } from "../../js/src/lib/interfaces/Types";

export interface ExampleRepo {
Expand Down Expand Up @@ -46,7 +46,7 @@ export interface TaskData {
id: PipelineType;
isPlaceholder?: boolean;
label: string;
libraries: Array<keyof typeof ModelLibrary>;
libraries: ModelLibraryKey[];
metrics: ExampleRepo[];
models: ExampleRepo[];
summary: string;
Expand Down
4 changes: 2 additions & 2 deletions tasks/src/const.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import type { ModelLibrary } from "../../js/src/lib/interfaces/Libraries";
import type { ModelLibraryKey } from "../../js/src/lib/interfaces/Libraries";
import type { PipelineType } from "../../js/src/lib/interfaces/Types";

/*
* Model libraries compatible with each ML task
*/
export const TASKS_MODEL_LIBRARIES: Record<PipelineType, Array<keyof typeof ModelLibrary>> = {
export const TASKS_MODEL_LIBRARIES: Record<PipelineType, ModelLibraryKey[]> = {
"audio-classification": ["speechbrain", "transformers"],
"audio-to-audio": ["asteroid", "speechbrain"],
"automatic-speech-recognition": ["espnet", "nemo", "speechbrain", "transformers"],
Expand Down