Skip to content

v0.2.20 - OptiLLM, Langfuse v3
Compare
Choose a tag to compare
@av av released this 29 Dec 12:01
· 35 commits to main since this release
v0.2.20
bcd254a

OptiLLM

optillm is an optimising LLM proxy, similar to Harbor Boost with a lot of advanced reasoning/planning workflows.

# Will build and start the service
# [--tail] is optional to automatically follow service logs after start
harbor up optillm --tail

optillm is connected to all inference backends in Harbor out of the box (but haven't been tested). See compatibility guide on making it work with Open WebUI.

Misc

  • langfuse was updated to v3
Assets 2
Loading