Ability to use Python LLM package plugins

[castedo]

This is a feature idea which I might add to copyaid.it. Might be something manubot-ai-editor wants to do too.

The llm Python package has a large number of plugins that can be used to easily (I assume) switch between different LLM APIs and local LLM models:

https://llm.datasette.io/en/stable/plugins/directory.html#plugin-directory

I imagine this plugin API interface is simple enough that it's a quick way to expand out into many different LLM implementations.

[miltondp]

Thank you, @castedo. We are currently implementing this feature but with LangChain. We haven't compared it with this package. Do you have any thoughts about this?

[castedo]

LangChain does much more than just make it easy to switch between LLM sources (i.e., OpenAI vs Anthropic vs local). Whereas calling the llm package plugins via whatever internal interface is used by llm is just one possible easy solution.

If you are using LangChain anyway for other reasons, then ease of LLM source switching is a nice secondary benefit that pops out and there is little reason to use the llm package interface.

But if all you want to achieve is ease of LLM source switching then LangChain is way overkill and will pull in a ton of dependencies (I think). llm is just a simple CLI tool kind of similar copyaid and manubot-ui-editor in CLI form.