Skip to content

Issues: modelscope/evalscope

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

25 Open 44 Closed
25 Open 44 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

模型性能压测-evalscope perf-报错超时
#177 opened Nov 4, 2024 by NiYueLiuFeng
3 tasks
@lxline
1
RAGAS评测- No module named 'evalscope.backend.rag_eval.utils
#170 opened Oct 30, 2024 by jackqdldd
1
@Yunnglin
35
Windows 11 下执行异常
#169 opened Oct 30, 2024 by Devliang24
9 tasks
@lxline
测试的结果数据统计好像有问题，为什么 首token 平均时间和每个包平均时延 平均时延 是一样？ duplicate This issue or pull request already exists
#168 opened Oct 30, 2024 by Devliang24
9 tasks
1
输出日志显示字符乱码，预期显示中文字符
#162 opened Oct 25, 2024 by Devliang24
4 of 11 tasks
2
数据库写入不进去
#150 opened Oct 12, 2024 by LeoCeasar
@lxline
22
选择winogrande测评报错
#148 opened Oct 11, 2024 by suchuxin
@wangxingjun778
4
基础模型评测（qwen2-7b-chat）报错
#147 opened Oct 11, 2024 by ljh567
2
模型性能测试报错
#144 opened Sep 29, 2024 by LHB-kk
@lxline
2
访问api 服务，当并发数大于等于500时，出现请求失败
#142 opened Sep 29, 2024 by LHB-kk
@lxline
1
测试结果指标数值相同
#141 opened Sep 29, 2024 by LHB-kk
@lxline
2
evalscope perf 测试sglang 部署的openai api server 无法输出结果
#128 opened Sep 12, 2024 by hetian127
@wangxingjun778
1
请问，评估支持使用昇腾910NPU嘛？
#123 opened Sep 9, 2024 by yiyayieryo
3
Baseline模型对比模式结果错误
#118 opened Aug 29, 2024 by stay-leave
2
请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？
#115 opened Aug 26, 2024 by EvilCalf
3
What if I set enable=false in evalscope/registry/config/cfg_single.yaml?
#106 opened Aug 15, 2024 by zhimin-z
1
调用OpenCompassBackendManager.list_datasets()错误
#105 opened Aug 15, 2024 by lyc0930
6
eval_swift_openai是否支持并发测试，怎么配置？
#95 opened Aug 9, 2024 by charliedream1
5
evalscope perf wandb
#94 opened Aug 7, 2024 by zll0000
2
evalscope perf --url 'our_url/v1/completions' --parallel 128 --model 'Qwen2-72B-Instruct' --log-every-n-query 10 --read-timeout=120 --dataset-path './data/open_qa.jsonl' -n 1 --max-prompt-length 128000 --api openai --stream --stop '<|im_end|>' --dataset openqa --debug
#93 opened Aug 7, 2024 by zll0000
7
OpenCompass Eval-Backend 不能用自定义的数据集吗？
#79 opened Jul 12, 2024 by jackqdldd
3
llmuses 0.3.2 执行自带的数据集报错：ImportError: cannot import name '_datasets_server' from 'datasets.utils' (/data/anaconda3/envs/eval-scope/lib/python3.10/site-packages/datasets/utils/__init__.py)
#76 opened Jul 11, 2024 by jackqdldd
12
perf流式输出报错
#64 opened Jul 1, 2024 by ccly1996
5
提供的本地数据集文件夹结构有误，无法直接使用
#39 opened Apr 23, 2024 by thewangcj
3
model_id为local ckpt路径时，infer template映射和解析出现问题
#24 opened Mar 26, 2024 by wangxingjun778
ProTip! Type g i on any issue or pull request to go back to the issue listing page.