Add descriptions and show defaults for all command line parameters #435
Conversation
| "auto_completions": list(known_models.keys()), | ||
| "description": "Choose from one of the supported models: " + ",".join( | ||
| list(known_models.keys()) | ||
| ), |
There was a problem hiding this comment.
I'd prefer not to put the models in the description. This description will make sense in the command line but not in the sphinx documentation. I prefer "The LLM used to respond to user requests and make edits."
There was a problem hiding this comment.
Description sounds good. Models, I feel, can be in the description since they are hard-coded in the code rather than being driven by a configuration.
There was a problem hiding this comment.
I agree with @jakethekoenig on this one; I would prefer not to put the list of models in the description. If argument parser has a way to add autocomplete, I would be ok with that.
| "The model used to prune the features before sending to the main model" | ||
| " for edits. Only used if llm_feature_filter is greater than 0." | ||
| " Possible models are: " | ||
| + ", ".join(list(known_models.keys())) |
There was a problem hiding this comment.
Again, would prefer not to have model names in description.
There was a problem hiding this comment.
Curious why the preference against listing the supported models? Its a common approach to list the supported "items" - e.g. pandoc documentation lists the supported file formats, etc. It also helps when using the options on the commandline and selecting a model without having to run the app first or read the code.
Perhaps better formatting on the list of models? The list is exactly the same as the autocomplete. In the case of generated documentation there is typically no autocomplete (at most you might have some generated javascript with the list hidden and then displayed for html version of the documentation).
There was a problem hiding this comment.
For one, there are going to be a lot of available models and I think it would bloat the description way too much. I'm working on switching us to a new library right now that can support models from any provider easily, and I don't want 10 or 20 lines of models in the descriptions.
There was a problem hiding this comment.
I was going to suggest moving to a lib like llmware that had a lot of other support (like vetors, local models etc.). I would change the description from the hardcoded list (which will hopefully e gone with the change) to one referencing the lib at that point.
mentat/config.py
Outdated
| validator=[validators.le(1), validators.ge(0)], | ||
| metadata={ | ||
| "description": ( | ||
| "The model's temperature. Temprature is a number between 0 to 1" |
There was a problem hiding this comment.
| "The model's temperature. Temprature is a number between 0 to 1" | |
| "The model's temperature. Temperature is a number between 0 to 1" |
Not all the commandline options had descriptions.
Also the defaults were not showing up in the documentation - fix that.