[WIP] LLaMa-Torchsharp for LLaMa 3

[GeorgeS2019]

@LittleLittleCloud
=> question; are there challenges that you find that prevented this project to continue?
https://github.com/LittleLittleCloud/Torchsharp-llama

Continue further:
@ejhg

did you add more beyond the llama 3 PR support by @LittleLittleCloud.
https://github.com/ejhg/llama-torchsharp

[LittleLittleCloud]

Found a better home for it. The project has migrated to Microsoft.ML.GenAI.Llama

• https://github.com/dotnet/machinelearning/tree/main/src/Microsoft.ML.GenAI.LLaMA

Sample: https://github.com/dotnet/machinelearning/tree/main/docs/samples/Microsoft.ML.GenAI.Samples

[GeorgeS2019]

LLaMaSharp is attempting to implement or has implemented many new features of LLama c++.

Do you have a future plan in this direction?

• Kernel Memory
• Grammar
• Template

[LittleLittleCloud]

Not really at the current time point, my current goal is to keep the ML.Net GenAI library as a pure torch implementation for the popular GenAI model and primarily focus on model training/fine tuning/inferencing. We have semantic kernel/autogen/M.E.A.I to support the GenAI usage cases that are not model inferencing

[ejhg]

@GeorgeS2019 I'd have to review @LittleLittleCloud 's latest code. If he already took care of Llama 3+, then nothing else in that direction.

Other than that, I made a contribution to TorchSharp.PyBridge that would reduce the memory overhead during load time allowing you to load slightly larger models faster. It'd be a matter of ensuring that that my fix made it into PyBridge and making sure TorchSharp llama is using it.