Skip to content

Issues: ModelTC/lightllm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
67 Open 128 Closed
67 Open 128 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

lightllm和vllm性能对比
#116 opened Sep 4, 2023 by Cydia2018
11
I tried test_llama.py，but.... help...T^T
#15 opened Aug 3, 2023 by MissQueen
4
调用报错
#16 opened Aug 4, 2023 by xxm1668
14
Auto convert without tokenizer.json to prevent performance downgrade?
#62 opened Aug 12, 2023 by leiwen83
2
Whether to support single card multi-instance loading
#66 opened Aug 14, 2023 by dushulin
4
Supporting GPT-NEO and GPT-NEOX enhancement New feature or request
#73 opened Aug 15, 2023 by goodbai-nlp
Which version is the comparison of the pressure measurement data provided in the project based on?
#78 opened Aug 16, 2023 by dushulin
5
是否有计划支持 lora？ enhancement New feature or request
#84 opened Aug 17, 2023 by Haskely
3
输出结果问题
#100 opened Aug 24, 2023 by listenyb
2
[FEATURE] Add support for true open-source models like MPT enhancement New feature or request
#102 opened Aug 24, 2023 by casper-hansen
1
baichuan13B load error
#106 opened Aug 28, 2023 by leiwen83
1
No performance gain with baichuan13B comparing with vllm
#107 opened Aug 28, 2023 by leiwen83
7
[feature request] add prompt styles support enhancement New feature or request
#9 opened Aug 3, 2023 by jiacheo
4
使用多卡加载模型，推理时报错
#109 opened Aug 29, 2023 by wx971025
24
Support generate() like vllm, and run without docker? enhancement New feature or request
#117 opened Sep 4, 2023 by curname
1
[BUG] chatglm-6b模型推理服务报错 bug Something isn't working
#121 opened Sep 6, 2023 by yeliang2258
5
[BUG]LLaMA2-7B服务首token延时异常 bug Something isn't working
#122 opened Sep 6, 2023 by yeliang2258
5
有类似chatgpt接口调用吗？ bug Something isn't working
#128 opened Sep 12, 2023 by xxm1668
6
[BUG] Issues ablout openai /v1/chat/completions interface in streaming mode bug Something isn't working enhancement New feature or request
#130 opened Sep 13, 2023 by sunxichen
2
[BUG]Qwen模型加载后NTK未生效 bug Something isn't working
#131 opened Sep 13, 2023 by xyfZzz
12
[Question] Flash attention only applies to prefilling stage bug Something isn't working
#147 opened Sep 24, 2023 by KexinFeng
2
[FEATURE] Load model directly from huggingface
#154 opened Sep 26, 2023 by Peilun-Li
[BUG]baichuan-13b-error bug Something isn't working
#156 opened Sep 28, 2023 by kuangdao
2
Quantization support
#163 opened Oct 16, 2023 by generalsvr
8
Question about Optimizations of Inference for batch_size = 1
#746 opened Feb 24, 2025 by OlinLai
1
ProTip! Mix and match filters to narrow down what you’re looking for.