Support for the phi-1.5 model in transformers.js

[bil-ash]

Model description

Please add support for the microsoft/phi-1.5 model to transformers.js. The model is really powerful when considering the size and should also fit in the 2gb model size limit. It is alo supported by optimum.

Prerequisites

• The model is supported in Transformers (i.e., listed here)
• The model can be exported to ONNX with Optimum (i.e., listed here)

Additional information

No response

Your contribution

none

[thekevinscott]

I'm trying to run this model via this URL.

In Chrome Version 120.0.6099.216 (Official Build) (arm64) I see the following for the sample code:

An error occurred during model execution: "RangeError: offset is out of bounds".

Models are being loaded off of Hugging Face. I'm on an M3 Max.

Code:

import { pipeline, env } from "@xenova/transformers";
env.allowLocalModels = false;

// Create a text-generation pipeline
const generator = await pipeline('text-generation', 'Xenova/phi-1_5_dev');

// Construct prompt
const prompt = `\`\`\`py
import math
def print_prime(n):
    """
    Print all primes between 1 and n
    """`;

// Generate text
const result = await generator(prompt, {
  max_new_tokens: 100,
});
console.log(result[0].generated_text);

Other models, e.g. Xenova/gpt2 work fine and produce output.

Also tested in Firefox, same error.

[xenova]

Hi there 👋 This is indeed a known issue (see #499), and will hopefully be fixed when we upgrade the onnxruntime-web version. It occurs for models >1B params. Running in node.js works correctly though.

[thekevinscott]

Ah gotcha, thanks @xenova ! I'll follow along there.