fix: instruction_template not applying from user_settings#4957
Conversation
|
Instruction templates are not set by name anymore. They are set by value in settings.yaml The idea is to have the templates in the model metadata, and in models/config.yaml otherwise. |
|
We could add button that would write the jinja script to the metadata if it is not already there. Once people start including the template in the metadata, all the problems would be solved. |
|
How to use and save custom template for gguf in this case? |
|
If the template is included in the GGUF metadata, it should be automatically set after #4975 Otherwise, the best option is to add a regex pattern to models/config.yaml or models/config-user.yaml (look at models/config.yaml for examples). |
The fact of the matter is that when you specify instruction template in models/config-user.yaml, the template is not loaded. This commit is precisely about the fact that config-user.yaml is loaded later than instruction_template_str is created |
|
instruction template is model metadata, not an user setting. |

Checklist: