Skip to content

Conversation

chuandudx
Copy link
Contributor

Fix the metric import path for new users who want to base their custom task on this template.

@clefourrier
Copy link
Member

It's likely not going to pass for style reasons as it's duplicating an import

@chuandudx
Copy link
Contributor Author

I fixed the duplicate import and ran ruff format according to the following - hope this works now

Run make quality
ruff format --check .
[7](https://github.com/huggingface/lighteval/actions/runs/10897826103/job/30244895251#step:5:8)8 files already formatted
ruff check .
community_tasks/_template.py:31:1: I001 [*] Import block is un-sorted or un-formatted
Found 1 error.
[*] 1 fixable with the `--fix` option.
make: *** [Makefile:[11](https://github.com/huggingface/lighteval/actions/runs/10897826103/job/30244895251#step:5:12): quality] Error 1
Error: Process completed with exit code 2.

Copy link
Member

@clefourrier clefourrier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, will merge if tests pass

…g automated tests for authentication purposes.
@chuandudx
Copy link
Contributor Author

chuandudx commented Sep 24, 2024

Hi Clémentine, I made a new commit (minor change in comments) in order to push code along with my HF_TOKEN since it appeared that the previous errors were related to user/password - would we be open to trying to submit again? Thank you!

Edit: Upon further investigation, I also set the HF_TOKEN in the environment secret of my fork of the repository. Maybe this will help?
image

@NathanHB
Copy link
Member

Hi ! You need a token that is able to access the https://huggingface.co/lighteval-tests org, I restarted the tests to see if it's because I need to be the one to trigger it

@chuandudx
Copy link
Contributor Author

Hi Nathan, thank you! Thanks for re-triggering it. Does the test check for the token from the person who triggered it or me as the submitter of the PR? Is there any action I need to take to obtain a token that has the appropriate access? Thank you!

Currently I have a user access token following these steps: https://huggingface.co/docs/hub/en/security-tokens

@clefourrier
Copy link
Member

@chuandudx You should not need any token on your side, it should be managed by our workflow entirely (and the token attached should be sent by the one triggering the pipeline).
We changed a small config thing recently and it broke this, so it's on us 😅 - normally one test should be skipped in the absence of the relevant token and it does not seem like it is.

@chuandudx
Copy link
Contributor Author

@clefourrier ohhh I see! Thank you for looking into this 😊!!

@clefourrier clefourrier merged commit 01e7240 into huggingface:main Oct 1, 2024
2 checks passed
@chuandudx chuandudx deleted the fix_template_import branch October 15, 2024 00:45
hynky1999 pushed a commit that referenced this pull request May 22, 2025
* Fix Metrics import path in community task template file.
---------

Co-authored-by: Clémentine Fourrier <[email protected]>
NathanHB pushed a commit that referenced this pull request Sep 19, 2025
* Fix Metrics import path in community task template file.
---------

Co-authored-by: Clémentine Fourrier <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants