Ollama: Forbidden (403) while fetching models on localhost

[sudhamjayanthi]

Describe the bug
Fails to pull models from Ollama server on the web version when clicking on fetch models button!

[Issue] Ollama: Forbidden (403) - is http://localhost:11434/api/tags accessible by the server?

Where is it happening?

• Which device: Desktop, Web version.
• Which browser: Arc
• Which website: https://get.big-agi.com/

To Reproduce
Steps to reproduce the behavior:

1. Run Ollama Server locally with OLLAMA_ORIGINS=https://get.big-agi.com/ ollama serve

   

2. Go to the web client.

3. Go to Models > Add > Ollama > Click on models button

   

Expected behavior
It pulls the following models from the server without any issues. (screenshot shows the JSON when I hit the same API endpoint in my browser)

Screenshots / context
Attached in relevant places above.

[enricoros]

Hi @sudhamjayanthi thanks for reporting - good for us to document this expected behavior.

The following is the issue:

• the server running on get.big-agi.com (a Vercel server on the cloud) is trying to connect to "localhost:11434"
• that cloud machine does not have an ollama running
• what the user wants is to connect on their own local machine, but that's inaccessible from the Vercel cloud

A quick schematic:

• Big-AGI Frontend, running on your Browser ✅ ---> big-AGI backend running on Vercel ✅ ---> "Localhost":11434 ❌ network inaccessible (as the error message also says)

A few ways to fix this issue:

• run big-AGI locally in production mode with npm i && npm run build && npm run start (see README.md for details)
  • this will make sure that both big-AGI and ollama are running on localhost and can talk to each other
• run big-AGI locally with Docker (and see our Docker Deployment documentation, to make sure it's on the host network)
• expose your localhost:11434 Ollama service to the internet (or make it accessible to the big-AGI server, and use a full IP address to reach it)
  • this depends on your OS/Networking skills, etc.

Let me know if this helps to fix this networking configuration issue.

Summary:

Edit: I've updated the Ollama Deployment docs docs/config-ollama.md to reflect this.

[enricoros]

Closing as an extensive doc is now provided in the app docs.

[Shadow-Wizard-Money-Gang]

i had the netwrok error previsouly on #276 and I tried their solutions and am now running big-AGI in a docker container and ollama locally, how do I connect them though as I am now getting a new error: [Issue] Ollama: (network) fetch failed - Error: connect ECONNREFUSED 127.0.0.1:11434