[WIP] Ollama support

[luisliz]

Not fully tested or working but at least this ran

Only changed this in .env

LLM_PROVIDER = "LOCAL" # "ANTHROPIC"
LOCAL_INFERENCE_MODEL="deepseek-v2"

[luisliz]

Some very simple benchmarking:

VM Specs: 3090-ti 24gb, threadripper (only 16 threads allocated) 32gb ram

Model	Parameters	VRam used	Results
Deepseek-v2	16b	11gb	Got very stuck on not generating good yaml for docs got up to db/schema. Generated PRD and FRD with some empty blocks.
Llama3.2	3.2B	4gb	Worked better than deepseek but also got stuck in generating yaml but did fill out better PRD and FRD.
dolphin-mixtral	8x7B	22gb	Works best in terms of generating docs but way slower and repeats a lot on yaml files. The sitemap was a total failure (left it 3 hours). This seems to be the best
deepseek-v2	16B	11gb	Not sure of code quality but SUPER fast in generating code. The schema gets cutoff seems to be too long causing no valid yaml found.

I'll post results if any of them finish anything.

[FeelsDaumenMan]

Could you make a tutorial on how to get this working?