Preset and doc for Mistral (multilingual)

[federicoparra]

Can't find preset for running Mistral, nor doc about how to.
In addition I see other models such as GPT2 have an English tokenizer preset whereas Mistral is trained to speak perfectly in many languages so the tokenizer should be multilingual not English.

[tirthasheshpatel]

Can't find preset for running Mistral, nor doc about how to.

Yeah, we haven't yet added the presets for Mistral. It will be added by the next release though.

In addition I see other models such as GPT2 have an English tokenizer preset whereas Mistral is trained to speak perfectly in many languages so the tokenizer should be multilingual not English.

Are the tokenizers for other languages published somewhere openly? The model raw file just contains a Sentence Piece model for english.

[RageshAntonyHM]

@tirthasheshpatel

what is the equivalent for this block in Mistral and Llama.

gpt2_preprocessor = keras_nlp.models.GPT2CausalLMPreprocessor.from_preset(
    "gpt2_base_en",
    sequence_length=256,
    add_end_token=True,
)
gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset("gpt2_base_en", preprocessor=gpt2_preprocessor)

Are they developed or not ?

[tirthasheshpatel]

Are they developed or not ?

Not yet, Mistral and LLaMA are still in development. The presets and a generator for both the models will be added by the next release!

[RageshAntonyHM]

@tirthasheshpatel

Thanks for reply .

Happy to hear.
Shall I know any roadmaps about the next release like tentative dates ?

[tirthasheshpatel]

There's no set/tentative release dates as far as I know. Maybe @mattdangerw knows more, but we will probably release the next version late February.

[mattdangerw]

We should be releasing next week, and we hope to have Mistral included in that release. Still tentative though.

[RageshAntonyHM]

@mattdangerw

Thanks.

I installed keras-nlp from your PR branch
!pip install -q git+https://github.com/tirthasheshpatel/keras-nlp.git@mistral-presets

When executing this block, I get this 403 error:

mistrl_preprocessor = MistralPreprocessor.from_preset(
    "mistral_7b_en",
    sequence_length=4096,
    add_end_token=True,
)

gpt2_lm = MistralCausalLM.from_preset("mistral_7b_en", preprocessor=mistrl_preprocessor)

Downloading from https://www.kaggle.com/api/v1/models/keras/mistral/keras/mistral_7b_en/1/download/tokenizer.json...
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/kagglehub/exceptions.py](https://localhost:8080/#) in kaggle_api_raise_for_status(response, resource_handle)
     48     try:
---> 49         response.raise_for_status()
     50     except requests.HTTPError as e:

10 frames
HTTPError: 403 Client Error: Forbidden for url: https://www.kaggle.com/api/v1/models/keras/mistral/keras/mistral_7b_en/1/download/tokenizer.json

The above exception was the direct cause of the following exception:

KaggleApiHTTPError                        Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/kagglehub/exceptions.py](https://localhost:8080/#) in kaggle_api_raise_for_status(response, resource_handle)
     69 
     70         # Default handling
---> 71         raise KaggleApiHTTPError(message, response=response) from e
     72 
     73 

KaggleApiHTTPError: 403 Client Error.

You don't have permission to access resource at URL: https://www.kaggle.com/models/keras/mistral/frameworks/keras/variations/mistral_7b_en/versions/1
Please make sure you are authenticated if you are trying to access a private resource or a resource requiring consent.

I installed kaggle package and also put the kaggle.json.

[RageshAntonyHM]

@tirthasheshpatel How to fix this permission error above ?

[tirthasheshpatel]

We are still testing the model, so weights are private. Once the PR is merged, you should be able to use the model.

[RageshAntonyHM]

@tirthasheshpatel

Yeah got it.

Is it possible to manually provide the model URL either from Kaggle or HuggingFace ?

[tirthasheshpatel]

Is it possible to manually provide the model URL either from Kaggle or HuggingFace ?

You should be able to convert the weights yourself from the huggingface model to the KerasNLP model using this script from my PR. But that requires having 80-100GB of RAM and I would just recommend waiting till we complete #1436

[RageshAntonyHM]

@tirthasheshpatel

Okay. No problem. Thanks