Emscripten build (demo, quick and dirty)

[ggerganov]

This is a very ugly hack to demonstrate build with Emscripten

Not intended to merge as it will just ruin the beauty of the implementation and most likely there is a much better way to do this. Mainly for educational purposes.

emcc -O3 run.c \
  -o web/llama2.js \
  -s EXPORTED_FUNCTIONS='["_main", "_main_loop", "_malloc", "_free"]' \
  -s EXPORTED_RUNTIME_METHODS='["ccall"]' \
  -s ALLOW_MEMORY_GROWTH=1 \
  --preload-file model.bin \
  --preload-file vocab.bin

The vocab.bin is generated in a similar way explained in #9

The produced artifacts will be generated in the web subfolder.

Example: https://ggerganov.com/llama2.c

[alexeykudinkin]

Worth merging IMO even if not for Emscripten, then for cleaning up the main sequence at the very least

[alexeykudinkin]

Suggested change

	printf("Unable to open file!");
	printf("Unable to open file!");
	return 1;

[alexeykudinkin]

Suggested change

	printf("Unable to open file!");
	printf("Unable to open file!");
	return 1;

[karpathy]

Very cool ty! I'm not as familiar with emscripten, I'll take some time to look at this and whether there is a way to support it gracefully.

[karpathy]

@ggerganov one thing I really like is the tokenizer inside C here, removing the need to use the run_wrap.py and sentencepiece. But it's not clear how you produced vocab.bin, presumably it's an export script?

[karpathy]

@ggerganov one thing I really like is the tokenizer inside C here, removing the need to use the run_wrap.py and sentencepiece. But it's not clear how you produced vocab.bin, presumably it's an export script?

nvm got it working in 3bfa566

[ggerganov]

Yes, for this PR I had exported the vocab from llama.cpp, which has been generated with a similar script as yours.

I'm not sure about the cause of the leading space - we have it in llama.cpp output as well and haven't dug to understand how to fix it yet.

[kroggen]

The tokenizer stores a leading space on some tokens.

In python we use the Decode function, that removes the leading space on the first token

But this repo is using the id_to_piece() function to export the tokenizer

python3
>>> import sentencepiece as spm
>>> sp = spm.SentencePieceProcessor(model_file='tokenizer.model')
>>> sp.id_to_piece([9038])
['▁Once']

Then it is simply printed, without considering if it is the first token:

llama2.c/run.c

Line 471 in f565089

printf("%s", vocab[next]);

The PR #89 fixes it

[gohai]

I recreated @ggerganov's changes against the current tree (with support for prompt): https://github.com/gohai/llama2.c/commits/emscripten

[rahuldshetty]

I've built a JS framework around this idea to run Language Models on Web.
This also leverages Emscripten to compile llama2.c into WASM JS.

llama2.c on Web Demo: https://rahuldshetty.github.io/ggml.js-examples/llama2_tinystories.html
ggml.js Framework: https://github.com/rahuldshetty/ggml.js