Skip to content

Conversation

clefourrier
Copy link
Member

See slack. Will need to be merged after the max length fix, but is reviewable.

"lighteval",
"original",
"extended",
"custom",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we keep custom ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Custom is for experiments you might want to do on your side, community for community added tasks imo

self, task_name_list: List[str], custom_tasks: Optional[Union[str, ModuleType]] = None
self,
task_name_list: List[str],
custom_tasks: Optional[Union[str, ModuleType]] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mhh so we keep both custom and extended ? Or custom here is meant to be community ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

community and custom tasks can be launched here

@clefourrier clefourrier merged commit df21407 into main Mar 9, 2024
hynky1999 pushed a commit that referenced this pull request May 22, 2025
* init - now gives the path with an arg, maybe will remove

* allows several custom task modules to be loaded

* fix quality

---------

Co-authored-by: Nathan Habib <[email protected]>
Co-authored-by: Nathan Habib <[email protected]>
NathanHB added a commit that referenced this pull request Sep 19, 2025
* init - now gives the path with an arg, maybe will remove

* allows several custom task modules to be loaded

* fix quality

---------

Co-authored-by: Nathan Habib <[email protected]>
Co-authored-by: Nathan Habib <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants