Update modal example #1111
Update modal example #1111
Conversation
|
Addressed the comments here -- any other comments? |
We generally pin to improve the stability of our examples. But you'll notice we don't pin Modal versions, because we expect to handle any issues that our own code changes cause. Makes sense for you to do the same!
I have mostly moved over to using forks or finetunes of gated models that are not themselves gated, e.g. from NousResearch, to avoid this bit of friction. |
|
Re: non-gated versions of models. I made that change to our version in this commit. Note that the |
- use ungated model - add sentencepiece dependency
|
Great, good idea! I added the ungated Mistral model, which also required a |
docs/cookbook/deploy-using-modal.md
Outdated
| ## Build the image | ||
|
|
||
| First we need to define our container image. We download the Mistral-7B-v0.1 model from HuggingFace as part of the definition of the image so it only needs to be done once (you need to provide an [access token](https://huggingface.co/settings/tokens)) | ||
| First we need to define our container image. We download the Mistral-7B-v0.1 model from HuggingFace as part of the definition of the image so it only needs to be done once (you need to provide an [access token](https://huggingface.co/settings/tokens)). |
rlouf
added
examples
I updated some of the text of the modal example to add more information on what's happening where, since Modal's general structure may be unfamiliar to new users.
List of changes:
import_modelto a.envcall when the image is created. This is more idiomatic Modal code. It was previously addressed in Documentation: Fix failing Modal example #1058, but this should be more current.