Skip to content

fix: instruction_template not applying from user_settings#4957

Closed
hronoas wants to merge 1 commit into
oobabooga:mainfrom
hronoas:patch-2
Closed

fix: instruction_template not applying from user_settings#4957
hronoas wants to merge 1 commit into
oobabooga:mainfrom
hronoas:patch-2

Conversation

@hronoas
Copy link
Copy Markdown
Contributor

@hronoas hronoas commented Dec 17, 2023

Checklist:

@oobabooga
Copy link
Copy Markdown
Owner

Instruction templates are not set by name anymore. They are set by value in settings.yaml

The idea is to have the templates in the model metadata, and in models/config.yaml otherwise.

@FartyPants
Copy link
Copy Markdown
Contributor

FartyPants commented Dec 17, 2023

We could add button that would write the jinja script to the metadata if it is not already there. Once people start including the template in the metadata, all the problems would be solved.
Or I could prob. make extension that does it.

@hronoas
Copy link
Copy Markdown
Contributor Author

hronoas commented Dec 18, 2023

How to use and save custom template for gguf in this case?
Settings.yaml - used globally, I would like to use an individual custom template for each model.
There are models in which a start prompt is prescribed. I prefer not to change the template manually every time.

@oobabooga
Copy link
Copy Markdown
Owner

If the template is included in the GGUF metadata, it should be automatically set after #4975

Otherwise, the best option is to add a regex pattern to models/config.yaml or models/config-user.yaml (look at models/config.yaml for examples).

@hronoas
Copy link
Copy Markdown
Contributor Author

hronoas commented Dec 18, 2023

Otherwise, the best option is to add a regex pattern to models/config.yaml or models/config-user.yaml (look at models/config.yaml for examples).

The fact of the matter is that when you specify instruction template in models/config-user.yaml, the template is not loaded. This commit is precisely about the fact that config-user.yaml is loaded later than instruction_template_str is created

@hronoas
Copy link
Copy Markdown
Contributor Author

hronoas commented Dec 18, 2023

изображение

@oobabooga oobabooga closed this Dec 18, 2023
@oobabooga
Copy link
Copy Markdown
Owner

instruction template is model metadata, not an user setting.

@hronoas hronoas deleted the patch-2 branch April 21, 2024 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants