Feature/add ability change temperature

[waynehamadi]

Background

Currently the temperature by default is None. None means 1 for GPT3.5. the value can go from :

• 0 => this means the llm is more likely to give the same answer given the same question, because it considers less options and goes for the most "likely" option. Less creativity here
• 2 => more creativity, but as a result can give answers that less close to "the optimal answer"

I personally love 0 temperature. It makes the agent consistent.

Changes

Super easy: environment variable and update config. I kept the default temperature to 1, but highly recommend 0.
In fact, I have seen way less json errors with 0.

Documentation

Test Plan

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes