Discussions

• 1 You must be logged in to vote

  💬

  FYI, Someone has finally managed to get modern AMD GPUs to attach to a Raspberry Pi and run LLMs through Vulkan

  sblaszak started Nov 20, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  AMD Performance (MI100) vs NV3090

  ccbadd asked Aug 16, 2023 in Q&A · Unanswered

  

  

  

  

  4
• 2 You must be logged in to vote

  🙏

  24GB 3090/4090 + 16GB Tesla P100 = 70B (almost)?

  EyeDeck asked Jul 28, 2023 in Q&A · Unanswered

  

  

  

  

  4
• 2 You must be logged in to vote

  💬

  The FALCON-180b is totally hot and sexy

  akaikite started Sep 10, 2023 in General

  

  

  2
• 2 You must be logged in to vote

  🙏

  Release a loaded model

  sammyf asked Oct 8, 2023 in Q&A · Closed · Answered

  

  

  3
• 1 You must be logged in to vote

  🙏

  How to decrease time to generate first token?

  VenkatLohithDasari asked Sep 26, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

  hiqsociety asked Sep 25, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Inference speed

  jchavezar asked Sep 21, 2023 in Q&A · Unanswered

  

  

  1
• 10 You must be logged in to vote

  💬

  Perf test on various HW

  dvoidus started May 29, 2023 in General

  

  

  

  

  44
• 1 You must be logged in to vote

  💬

  Question about inference speed

  MrJungle1 started Jul 10, 2023 in General

  

  

  

  

  27
• 1 You must be logged in to vote

  🙏

  ran docker example but got No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' error

  hiqsociety asked Sep 14, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Cash grant?

  TikkunCreation started Sep 1, 2023 in General

  

  

  

  

  6
• 3 You must be logged in to vote

  🙏

  Multiple safetensor files, am I missing something?

  mousepixels asked Jul 20, 2023 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  web ui from another machine

  ccbadd asked Aug 14, 2023 in Q&A · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  jllllll/exllama

  techbutler asked Aug 9, 2023 in Q&A · Unanswered

  

  

  3
• 3 You must be logged in to vote

  💬

  Llama 2 is out

  nikshepsvn started Jul 18, 2023 in General

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  Any info on the tuning?

  Ph0rk0z asked Jul 16, 2023 in Q&A · Unanswered

  

  

  4
• 1 You must be logged in to vote

  💬

  exllama + triton inference server?

  arbi-dev started Jul 31, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  How can I assign GPU allocation?

  nivibilla asked Jul 28, 2023 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  CUDA error: no kernel image is available for execution on the device

  NigelNelson asked Jul 25, 2023 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  💡

  Cache Hidden State

  kaiokendev started Jul 25, 2023 in Ideas

  

  

  7
• 1 You must be logged in to vote

  💬

  How does turbodep go about optimising performance?

  ri938 started Jul 24, 2023 in General

  

  

  1
• 1 You must be logged in to vote

  💬

  Best alpha/cpe values while using extended context length on Exllama?

  nikshepsvn started Jul 23, 2023 in General

  

  

  1
• 1 You must be logged in to vote

  💡

  Have you tried moving MLP layer to CPU RAM

  kaiokendev started Jul 22, 2023 in Ideas · Closed

  

  

  9
• 1 You must be logged in to vote

  🙏

  Disabling cache

  psinger asked Jul 21, 2023 in Q&A · Unanswered

  

  

  1