Skip to content

Issues: turboderp/exllamav2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

76 Open 382 Closed
76 Open 382 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[QUESTION] Does exllamav2 support no-dequant inference?
#670 opened Nov 7, 2024 by AaronZLT
1
[REQUEST] Synthetic Data generation features
#669 opened Nov 3, 2024 by AstrisCantCode
3 tasks done
1
[PAPER] New quant method with SOTA quality and speed: QTIP
#668 opened Nov 1, 2024 by TyraVex
3 tasks done
[BUG] How can we increase or reduce the cache size bug Something isn't working
#665 opened Oct 29, 2024 by royallavanya140
3 tasks done
1
[REQUEST] Alternative way to the Pytorch environment variables on Windows to set Pytorch memory management parameters
#664 opened Oct 29, 2024 by Nexesenex
3 tasks done
5
[BUG] AMD - Out of memory errors despite having plenty of VRAM bug Something isn't working
#662 opened Oct 27, 2024 by RSAStudioGames
3 tasks done
[REQUEST] Faster 6/8-bit EXL2 quantization
#660 opened Oct 19, 2024 by grimulkan
3 tasks done
[REQUEST] Llama 3.2 Vision Support (or already exists?)
#658 opened Oct 18, 2024 by grimulkan
3 tasks done
12
[BUG] Appending-Runtime-LoRA-weights bug Something isn't working
#656 opened Oct 16, 2024 by royallavanya140
3 tasks done
1
[BUG] Convert script fails to run on master branch as of v0.2.3 bug Something isn't working
#655 opened Oct 15, 2024 by iamwavecut
3 tasks done
5
[BUG] RAM UTILISATION IS INCREASING RAPIDLY bug Something isn't working
#639 opened Sep 25, 2024 by UTSAV-44
1
[REQUEST] Is it possible and a lot of trouble to support flux?
#631 opened Sep 22, 2024 by Ph0rk0z
3 tasks done
4
[BUG] Random slowdowns in tensor parallel. bug Something isn't working
#630 opened Sep 21, 2024 by Ph0rk0z
3 tasks done
2
[BUG] Failed to quantize Qwen2.5-Math-72B-Instruct: Measurement/inference error (3): hidden_states bug Something isn't working
#627 opened Sep 19, 2024 by Orion-zhen
3 tasks done
7
[BUG] Quantization of Qwen return garbage bug Something isn't working
#621 opened Sep 10, 2024 by fahadh4ilyas
3 tasks done
9
How to implement paged attention in HF format?
#616 opened Sep 6, 2024 by fahadh4ilyas
5
Want to try row split + all_reduce for MLP and attn
#614 opened Sep 3, 2024 by Azure-Tang
4
how can i solve this problem
#611 opened Sep 2, 2024 by Sultan0ML
Async Stream Genenerator?
#604 opened Aug 28, 2024 by KingBipo
3
Does NVLink improve tensor parallelism?
#603 opened Aug 26, 2024 by bryanhpchiang
1
## Measurement/inference error (3): hidden_states
#600 opened Aug 24, 2024 by IllIIllIlllIIl
6
Do you know of any code framework that supports fast attention score calculation similar to flash attention?
#599 opened Aug 24, 2024 by qiyuxinlin
Tensor parallelism issues
#598 opened Aug 24, 2024 by dirkson
5
Request for multi model support
#595 opened Aug 20, 2024 by royallavanya140
1
[qesstion] Wrapper Linear API and 2bits
#589 opened Aug 14, 2024 by wenhuach21
4
ProTip! Find all open issues with in progress development work with linked:pr.