v0.1.25
v0.1.25 - KTransformers integration
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
🔥 Show Cases | 🚀 Quick Start | 📃 Tutorial | 💬 DiscussionStarting
# [Optional] Pre-build the image
# This is very large, as it's based on pytorch+cuda
# go grab a coffee!
harbor build ktransformers
# Start the service
harbor up ktransformers
Harbor's version was monkey-patched to be compatible with Open WebUI and will appears as ktransformers
in the model selector upon successful start.
https://github.com/av/harbor/wiki/ktransformers-webui.png
Full Changelog: v0.1.24...v0.1.25