🏫🙋🏻Self Learn LLMs

This repository constitutes some of the resources which I will use to learn about Large Language Models. I will also try to come up with a roadmap as I go forward in this self-learning journey, since a clear roadmap with milestones will be one of the best ways to learn about LLMs in a proper manner.

For this, I will include a mix of theoretical and practical hands-on resources to learn.

PS: Need to make this more visual

Edit: 6th Nov 2024

• Need to majorly refactor the repository.
• Remove courses or resources that are not relevant.
• In the model race, we can't go on listing models in the repository as it's tough to keep track of models and updates will make the previous models useless. Need to think of a better way to organize model zoo.
• Maybe add 1-2 contributors or open the repository to contributions to help out.
• How can we make it a gret learning experience, hosting notes and tutorials, open discussions, webpage?

Courses

📝🗒️ NLP Fundamentals

1. CS224N Natural Language Processing with Deep Learning, Stanford
2. Natural Language Processing Specialization, Coursera

🤖 Transformers

1. HuggingFace NLP + Transformers Course
2. CS25: Transformers United V2, Stanford CS25, Fall 2021 Version

🦜🗣️ Large Language Models

Industrial and Open-Source courses

1. Activeloop Learn, this initiative GenAI360 provides 3 free courses on RAGs, fine-tuning LLMs, LangChain and VectorDBs.
2. LLM Course by Maxime Labonne, Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
3. Hands on LLM Course, Learn about LLMs, LLMOps, and Vector DBs for free by designing, training, and deploying a real-time financial advisor LLM system source code + video & reading materials.
4. Full Stack Deep Learning, started out as a deep learning bootcamp and evolved into LLM bootcamp around April 2023, now is free to take up.
5. LLM University by Cohere, this course consists of 8 modules taught by the famous Luis Serrano, who is known for teaching concepts in a easy and visually appealing manner. The course contains topics like fundamentals, deployment, semantic search and RAG.
6. Deeplearning.ai Short Courses, Short courses by DL.AI on various domains of LLMs and Generative AI. These short courses are really useful as they have the perfect blend of theoretical and practical sessions. The courses are usually made in collaborations with companies like Hugging Face, Mistral, OpenAI, Microsoft, Meta, Google etc.
7. LLM Zoomcamp by DataTalksClub, LLM Zoomcamp - a free online course about building a Q&A system.
8. Applied LLMs Mastery 2024 Course by Aishwarya N Reganti, free 10 weeks course with a definite roadmap ranging from LLM Fundamentals, Tools and techniques, Deployment and evaluation to Challenges and future trends.
9. Weights and Biases Courses, provides different courses on MLOps, LLM Powered Apps etc.
10. LLM Models course, DataBricks x ed, professional certification by DataBricks.
11. Deeplearning.ai offers various short courses on LLMs like LangChain for LLM App Development, Serverless LLMs with AWS Bedrock, Fine-tuning LLMs, LLMs with Semantic Search etc.
12. Introduction to Generative AI Learning Path, Google Cloud.
13. Arize University hosts courses like llm-evaluation, llm agents tools and chains, llm-observability etc.

University Courses

1. CS 324, Stanford
2. COMP790-101: Large Language Models, UNC Chapel Hill
3. Large Language Models S-23, ETH Zurich
4. Foundations of Large Language Models, University of Waterloo

📕📗📙 Books

1. Natural Language Processing with Transformers Book, by Lewis Tunstall, Leandro von Werra and Thomas Wolf | Notebooks for the book
2. Build a Large Language Model (From Scratch), by Sebastian Raschka | Official Code Repository for the book
3. LLM Engineer's Handbook_ Master the art of engineering large language models from concept to production, by Paul Iusztin and Maxime Labonne | Code for the book
4. Hands-On Large Language Models: Language Understanding and Generation, by Jay Alammar and Maarten Grootendorst | Code repository for the book
5. AI Engineering: Building Applications with Foundation Models, by Chip Huyen | Resources for the book

✍🏻🛣️ Blogs and Guides

Langchain Blogs

1. AIMultiple's blog on Large Language Models: Complete Guide in 2023
2. Cohere Docs
3. FutureSmart AI Blog on Building Chatbots using LangChain and ChatGPT
4. Task-driven Autonomous Agent Utilizing GPT-4, Pinecone, and LangChain for Diverse Applications

📰 Newsletters

👨🏻‍💻👨🏻‍💻 Tutorials

1. Let's build GPT: from scratch, in code, spelled out.: 4 hour long YouTube video by Andrej Karpathy on building GPT from scratch.
2. [1hr talk] Intro to Large Language Models: 1 hour YouTube video by Andrej Karpathy on Introduction to LLMs.
3. Let's build the GPT Tokenizer: 2 hour long YouTube video by Andrej Karpathy on building the GPT Tokenizer.
4. Let's reproduce GPT-2 (124M): 4 hour long YouTube Video by Andrej Karpathy on reproducing GPT-2 (124M model).

Libraries, Frameworks and Toolkits

// Pipeline: Data Collection (Scraping) and Datasets, Pre-Training, Fine-Tuning, Deployment, Evaluations, Security, Guardrails.

1. LLMStudio, H2O AI
2. LLamaIndex
3. NeMo Guardrails, NVIDIA, to prevent hallucinations and add programmable guardrails
4. MLC LLM, Develop optimize and deploy LLMs natively on everyone's devices)
5. LaMini LLM

🌍🌎🌏 Communities

Usecases

1. ShareGPT

Projects and Ideas

🧑🏻‍🤝‍🧑🏻 People to Follow

People you should definitely follow to keep updated about LLMs. Researchers/Founders/Developers/AI Content Creators involved in LLM production/research/development

1. Sebastian Raschka, he is a legend and will burst your hype-up LLM bubble with his amazing tweets, blogs and tutorials. Subscribe to his newsletter Ahead of AI
2. Andrej Karpathy, so this legend worked in Tesla, took a break, started his YouTube channel to teach the fundamentals and blew us all with his amazing video on implementing GPT from scratch and finally rejoined OpenAI. I guess you cannot lose a legend :D
3. Jay Alammar, yup if you don't know about his ELI blog on Transformers go read that out first and be sure to follow him for updates.
4. Tomaz Bratanic, he is the author of famous book Graph Algorithms for Data Science, and currently writes great blogs on Medium related to GPT, Langchain and stuff.
5. Maxime Labonne
6. Paul Iusztin