WebGPU LLM Loader

Getting Started

First, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

You can choose to load a pre-built Llama 2 Model(7B, 13B, 70B) or a Custom Model that you build yourself.

If you select a Custom LLM Model, three inputs are required.

• Model Local Id: The local_id from your mlc-chat-config.json.
• Model URL: The URL of the Model you uploaded to huggingface.
• WASM File: The .wasm file built for your WebGPU.

Learn More

To learn more about Next.js, take a look at the following resources:

• Compile Models via MLC
• Distribute Compiled Models

You can check out GitHub repository - your feedback and contributions are welcome!