Within this segment of our course, we will delve into the latest research developments surrounding LLMs. Kicking off with an examination of MultiModal Large Language Models (MM-LLMs), we'll explore how this particular area is advancing swiftly. Following that, our discussion will extend to popular open-source models, focusing on their construction and contributions. Subsequently, we'll tackle the concept of agents that possess the capability to carry out tasks autonomously from inception to completion. Additionally, we'll understand the role of domain-specific models in enriching specialized knowledge across various sectors and take a closer look at groundbreaking architectures such as the Mixture of Experts and RWKV, which are set to improve the scalability and efficiency of LLMs.
In the past year, there have been notable advancements in MultiModal Large Language Models (MM-LLMs). Specifically, MM-LLMs represent a significant evolution in the space of language models, as they incorporate multimodal components alongside their text processing capabilities. While progress has also been made in multimodal models in general, MM-LLMs have experienced particularly substantial improvements, largely due to the remarkable enhancements in LLMs over the year, upon which they heavily rely.
Moreover, the development of MM-LLMs has been greatly aided by the adoption of cost-effective training strategies. These strategies have enabled these models to efficiently manage inputs and outputs across multiple modalities. Unlike conventional models, MM-LLMs not only retain the impressive reasoning and decision-making capabilities inherent in Large Language Models but also expand their utility to address a diverse array of tasks spanning various modalities.
To understand how MM-LLMs function, we can go over some common architectural components. Most MM-LLMs can be divided in 5 main components as shown in the image below. The components explained below are adapted from the paper “MM-LLMs: Recent Advances in MultiModal Large Language Models”. Let’s understand each of the components in detail.
Image Source: https://arxiv.org/pdf/2401.13601.pdf
1. Modality Encoder: The Modality Encoder (ME) plays a pivotal role in encoding inputs from diverse modalities
2. Input Projector: The Input Projector
3. LLM Backbone: The LLM Backbone serves as the core agent in MM-LLMs, inheriting notable properties from LLMs such as zero-shot generalization, few-shot In-Context Learning (ICL), Chain-of-Thought (CoT), and instruction following. The backbone processes representations from various modalities, engaging in semantic understanding, reasoning, and decision-making regarding the inputs. Additionally, some MM-LLMs incorporate Parameter-Efficient Fine-Tuning (PEFT) methods like Prefix-tuning, Adapter, or LoRA to minimize the number of additional trainable parameters.
4. Output Projector: The Output Projector
5. Modality Generator: The Modality Generator
MM-LLMs are trained in two main stages: MultiModal Pre-Training (MM PT) and MultiModal Instruction-Tuning (MM IT).
MM PT: During MM PT, MM-LLMs are trained to understand and generate content from different types of data like images, videos, and text. They learn to align these different kinds of information to work together. For example, they learn to associate a picture of a cat with the word "cat" and vice versa. This stage focuses on teaching the model to handle different types of input and output.
MM IT: In MM IT, the model is fine-tuned based on specific instructions. This helps the model adapt to new tasks and perform better on them. There are two main methods used in MM IT:
- Supervised Fine-Tuning (SFT): The model is trained on examples that are structured in a way that includes instructions. For instance, in a question-answer task, each question is paired with the correct answer. This helps the model learn to follow instructions and generate appropriate responses.
- Reinforcement Learning from Human Feedback (RLHF): The model receives feedback on its responses, usually in the form of human-generated feedback. This feedback helps the model improve its performance over time by learning from its mistakes.
Therefore MM-LLMs are trained to understand and generate content from multiple sources of information, and they can be fine-tuned to perform specific tasks better based on instructions and feedback.
The below diagram summarizes popular MM-LLMs and models used for each of their components.
Image Source: https://arxiv.org/pdf/2401.13601.pdf
Some potential future directions for MM-LLMs involve extending their capabilities through various avenues:
- More Powerful Models:
- Extend MM-LLMs to accommodate additional modalities beyond the current ones like image, video, audio, 3D, and text, such as web pages, heat maps, and figures/tables.
- Incorporate various types and sizes of LLMs to provide practitioners with flexibility in selecting the most suitable one for their specific requirements.
- Enhance MM IT datasets by diversifying the range of instructions to improve MM-LLMs' understanding and execution of user commands.
- Explore integrating retrieval-based approaches to complement generative processes in MM-LLMs, potentially enhancing overall performance.
- More Challenging Benchmarks:
- Develop larger-scale benchmarks that include a wider range of modalities and use unified evaluation standards to adequately challenge the capabilities of MM-LLMs.
- Tailor benchmarks to assess MM-LLMs' proficiency in practical applications, such as evaluating their ability to discern and respond to nuanced aspects of social abuse presented in memes.
- Mobile/Lightweight Deployment:
- Develop lightweight implementations to deploy MM-LLMs on resource-constrained platforms like low-power mobile and IoT devices, ensuring optimal performance.
- Embodied Intelligence:
- Explore embodied intelligence to replicate human-like perception and interaction with the surroundings, enabling robots to autonomously implement extended plans based on real-time observations.
- Further enhance MM-LLM-based embodied intelligence to improve the autonomy of robots, building on existing advancements like PaLM-E and EmbodiedGPT.
- Continual IT:
- Develop approaches for MM-LLMs to continually adapt to new MM tasks while maintaining superior performance on previously learned tasks, addressing challenges such as catastrophic forgetting and negative forward transfer.
- Establish benchmarks and develop methods to overcome challenges in continual IT for MM-LLMs, ensuring efficient adaptation to emerging requirements without substantial retraining costs.
Recent developments in open-source LLMs have been pivotal in democratizing access to advanced AI technologies. Open-source LLMs offer several advantages over closed-source models, enhancing transparency, customizability, and collaboration. They allow for a deeper understanding of model workings, enable modifications to suit specific needs, and encourage improvements through community contributions. They also serve as educational tools and support a diverse AI ecosystem, preventing monopolies. However, challenges such as computational demands and potential misuse exist, but the benefits of open-source models often outweigh these issues, especially for those valuing openness and adaptability in AI development.
A few popular Open-Source LLMs are listed below:
- LLaMA (13B parameters) was released by Meta in February 2023, outperforming GPT-3 on many NLP benchmarks despite having fewer parameters. LLaMA-2, an enhanced version with 40% more data and doubled context length, was released in July 2023 along with specialized versions for conversations (LLaMA 2-Chat) and code generation (LLaMA Code).
- Developed by a Paris-based startup, Mistral 7B set new benchmarks by outperforming all existing open-source LLMs up to 13B parameters in English and code benchmarks. Mistral AI later also released Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) model. This model marks a departure from traditional AI architectures and training methods, aiming to provide the developer community with innovative tools that can inspire new applications and technologies. We’ll learn more about the Mixture of Experts paradigm in the next serction
- OLMo is part of the AI2 LLM framework aimed at encouraging open research by providing access to training data, code, models, and evaluation tools. It includes the Dolma dataset, comprehensive training and inference code, model weights for four 7B scale variants, and an extensive evaluation suite under the Catwalk project.
- LLM360 proposes a fully open-source approach to LLM development, advocating for the release of training code, data, model checkpoints, and intermediate results. It released two 7B parameter LLMs, AMBER and CRYSTALCODER, complete with resources for transparency and reproducibility in LLM training.
While Llama and Mistral only release their models, OLMo and LLM360 go further by providing checkpoints, datasets, and more, ensuring their offerings are fully open and capable of being reproduced.
LLM Agents have been gaining significant momentum in recent months and represent the future and expansion of LLM capabilities. An LLM agent is an AI system that employs a large language model at its core to perform a wide range of tasks, not limited to text generation. These tasks include conducting conversations, reasoning, completing various tasks, and exhibiting autonomous behaviors based on the context and instructions provided. LLM agents operate through sophisticated prompt engineering, where instructions, context, and permissions are encoded to guide the agent's actions and responses.
- Autonomy: LLM agents can operate with varying degrees of autonomy, from reactive to proactive behaviors, based on their design and the prompts they receive.
- Task Completion: With access to external knowledge bases, tools, and reasoning capabilities, LLM agents can assist in or independently handle a variety of applications, from chatbots to complex workflow automation.
- Adaptability: Their language modeling strength allows them to understand and follow natural language prompts, making them versatile and capable of customizing their responses and actions.
- Advanced Skills: Through prompt engineering, LLM agents can be equipped with advanced analytical, planning, and execution skills. They can manage tasks with minimal human intervention, relying on their ability to access and process information.
- Collaboration: They enable seamless collaboration between humans and AI by responding to interactive prompts and integrating feedback into their operations.
LLM agents combine the core language processing capabilities of LLMs with additional modules like planning, memory, and tool usage, effectively becoming the "brain" that directs a series of operations to fulfill tasks or respond to queries. This architecture allows them to break down complex questions into manageable parts, retrieve and analyze relevant information, and generate comprehensive responses or visual representations as needed.
Example:
Suppose we're interested in organizing an international conference on sustainable energy solutions, aiming to cover topics such as renewable energy technologies, sustainability practices in energy production, and innovative policies for promoting green energy. The task involves complex planning and information gathering, including identifying key speakers, understanding current trends in sustainable energy, and engaging with stakeholders.
To tackle this multifaceted project, an LLM agent could be employed to:
- Research and Summarization: Break down the task into sub-tasks such as identifying emerging trends in sustainable energy, locating leading experts in the field, and summarizing recent research findings. The agent would use its access to a vast range of digital resources to compile comprehensive reports.
- Speaker Engagement: Draft personalized invitations to potential speakers, incorporating details about the conference's aims and how their expertise aligns with its goals. The agent can generate these communications based on profiles and previous works of the experts.
- Logistics Planning: Create a detailed plan for the conference, including a timeline of activities leading up to the event, a checklist for logistical arrangements (venue, virtual platform setup for hybrid participation, etc.), and a strategy for participant engagement. The agent can outline these plans by accessing databases of event planning resources and best practices.
- Stakeholder Communication: Draft updates and newsletters for stakeholders, providing insights into the conference's progress, highlights of the agenda, and key speakers confirmed. The agent tailors each communication piece to its audience, whether it's sponsors, participants, or the general public.
- Interactive Q&A Session Planning: Develop a framework for an interactive Q&A session, including pre-gathering questions from potential attendees, categorizing them, and preparing briefing documents for speakers. The agent can facilitate this by analyzing registration data and submitted queries.
In this scenario, the LLM agent not only aids in the execution of complex and time-consuming tasks but also ensures that the planning process is thorough, informed by the latest developments in sustainable energy, and tailored to the specific goals of the conference. By leveraging external databases, tools for data analysis and visualization, and its innate language processing capabilities, the LLM agent acts as a comprehensive assistant, streamlining the organization of a large-scale event with numerous moving parts.
The framework for LLM agents can be conceptualized through various lenses, and one such perspective is offered by the paper “A Survey on Large Language Model based Autonomous Agents”, through its distinctive components. This architecture is composed of four key modules: the Profiling Module, Memory Module, Planning Module, and Action Module. Each of these modules plays a crucial role in enabling the LLM agent to act autonomously and effectively in various scenarios.
Image Source : https://arxiv.org/pdf/2308.11432.pdf
- Profiling Module
The Profiling Module is responsible for defining the agent's identity and role. It incorporates information such as age, gender, career, personality traits, and social relationships to shape the agent's behavior. This module uses various methods to create profiles, including handcrafting for precise control, LLM-generation for scalability, and dataset alignment for real-world accuracy. The agent's profile significantly influences its interactions, decision-making processes, and the way it executes tasks, making this module foundational to the agent's design.
2. Memory Module
The Memory Module stores information the agent perceives from its environment and uses this stored knowledge to inform future actions. It mimics human memory processes, with structures inspired by sensory, short-term, and long-term memory. This module enables the agent to accumulate experiences, evolve based on past interactions, and behave in a consistent and effective manner. It ensures that the agent can recall past behaviors, learn from them, and adapt its strategies over time.
3. Planning Module
The Planning Module empowers the agent with the ability to decompose complex tasks into simpler subtasks and address them individually, mirroring human problem-solving strategies. It includes planning both with and without feedback, allowing for flexible adaptation to changing environments and requirements. Strategies such as single-path reasoning and Chain of Thought (CoT) are used to guide the agent in a step-by-step manner towards achieving its goals, making the planning process critical for the agent's effectiveness and reliability.
4. Action Module
The Action Module translates the agent's decisions into specific outcomes, directly interacting with the environment. It considers the goals of the actions, how actions are generated, the range of possible actions (action space), and the consequences of these actions. This module integrates inputs from the profiling, memory, and planning modules to execute decisions that align with the agent's objectives and capabilities. It is essential for the practical application of the agent's strategies, enabling it to produce tangible results in the real world.
Together, these modules form a comprehensive framework for LLM agent architecture, allowing for the creation of agents that can assume specific roles, perceive and learn from their environment, and autonomously execute tasks with a degree of sophistication and flexibility that mimics human behavior.
- Most LLM Agent research has been confined to text-based interactions. Expanding into multi-modal environments, where agents can process and generate outputs across various formats like images, audio, and video, introduces complexities in data processing and requires agents to interpret and respond to a broader range of sensory inputs.
- Hallucination, where models generate factually incorrect text, becomes more problematic in LLM agent systems due to the potential for cascading misinformation. Developing strategies to detect and mitigate hallucinations involves managing information flow to prevent inaccuracies from spreading across the network.
- While LLM agents learn from instant feedback, creating reliable interactive environments for scalable learning poses challenges. Furthermore, current methods focus on adjusting agents individually, not fully leveraging the collective intelligence that could emerge from coordinated interactions among multiple agents.
- Scaling the number of agents (multi-agent systems) for a use-case raises significant computational demands and complexities in coordination and communication among agents. Developing efficient orchestration methodologies is essential for optimizing workflows and ensuring effective multi-agent cooperation.
- Current benchmarks may not adequately capture the emergent behaviors critical to agents or span across diverse research domains. Developing comprehensive benchmarks is crucial for assessing agents’ capabilities in various fields, including science, economics, and healthcare.
While general LLMs are versatile and perform well on a broad range of tasks, they often fall short when it comes to handling specialized or niche tasks due to a lack of training on domain-specific data. Additionally, running these generic models can be costly. In these scenarios, domain-specific LLMs emerge as a superior alternative. Their training is focused on data from specific fields, which enhances their accuracy and provides them with a deeper understanding of the relevant terminology and concepts. This tailored approach not only improves their performance on tasks specific to a certain domain but also minimizes the chances of generating irrelevant or incorrect information.
Designed to adhere to the regulatory and ethical standards of their respective domains, these models ensure the appropriate handling of sensitive data. They also communicate more effectively with domain experts, thanks to their command of professional language. From an economic standpoint, domain-specific LLMs offer more efficient solutions by eliminating the need for significant manual adjustments. Furthermore, their specialized knowledge base enables the identification of unique insights and patterns, driving innovation in their respective fields.
Some popular domain specific LLMs are listed below
Clinical and Biomedical LLMs
- BioBERT: A domain-specific model pre-trained on large-scale biomedical corpora, designed to mine biomedical text effectively.
- Hi-BEHRT: Offers a hierarchical Transformer-based structure for analyzing extended sequences in electronic health records, showcasing the model's ability to handle complex medical data.
LLMs for Finance
- BloombergGPT: A finance-specific model with 50 billion parameters, trained on a vast array of financial data, showing excellence in financial tasks.
- FinGPT: A financial model fine-tuned with specific applications in mind, leveraging pre-existing LLMs for enhanced financial data understanding.
Code-Specific LLMs
- WizardCoder: Empowers Code LLMs with complex instruction fine-tuning, showcasing adaptability to coding domain challenges.
- CodeT5: A unified pre-trained model focusing on the semantics conveyed in code, highlighting the importance of developer-assigned identifiers in understanding programming tasks.
These domain-specific LLMs illustrate the vast potential and adaptability of AI across different fields, from understanding multilingual content and processing clinical data to financial analysis and code generation. By honing in on the unique challenges and data types of each domain, these models open up new avenues for innovation, efficiency, and accuracy in AI applications.
- Domain-specific LLMs will likely evolve to handle not just text but also images, audio, and other data types, enabling more comprehensive understanding and interaction capabilities across various formats.
- Future models may incorporate advanced interactive learning techniques, enabling them to update their knowledge base in real-time based on user feedback and new data, ensuring their outputs remain relevant and accurate.
- We might see an increase in systems where domain-specific LLMs work in concert with other AI technologies, such as decision-making algorithms and predictive models, to provide holistic solutions (Agents, like we discussed in the previous section)
- With growing awareness of AI's societal impact, the development of domain-specific LLMs will likely emphasize ethical considerations, fairness, and transparency, particularly in sensitive areas like healthcare and finance.
Mixture of Experts (MoEs) represents a sophisticated architecture within the realm of transformer models, focusing on enhancing model scalability and computational efficiency. Here's a breakdown of what MoEs are and their significance:
Definition and Components
- MoEs in Transformers: In transformer models, MoEs replace traditional dense feed-forward network (FFN) layers with sparse MoE layers. These layers comprise a number of "experts," each being a neural network—typically FFNs, but potentially more complex structures or even hierarchical MoEs.
- Experts: These are specialized neural networks (often FFNs) that handle specific portions of the data. An MoE layer may contain several experts, such as 8, allowing for a diverse range of data processing capabilities within the same model layer.
- Gate Network/Router: This is a critical component that directs input tokens to the appropriate experts based on learned parameters. The router decides, for instance, which expert is best suited to process a given input token, thus enabling a dynamic allocation of computational resources.
Advantages
- Efficient Pretraining: By utilizing MoEs, models can be pretrained with significantly less computational resources, allowing for larger model or dataset scales within the same compute budget as a dense model.
- Faster Inference: Despite having a large number of parameters, MoEs only use a subset for inference, leading to quicker processing times compared to dense models with a similar parameter count. However, this efficiency comes with the caveat of high memory requirements due to the need to load all parameters into RAM.
Challenges
- Training Generalization: While MoEs are more compute-efficient during pretraining, they have historically faced challenges in generalizing well during fine-tuning, often leading to overfitting.
- Memory Requirements: The efficient inference process of MoEs requires substantial memory to load the entire model's parameters, even though only a fraction are actively used during any given inference task.
Implementation Details
- Parameter Sharing: Not all parameters in a MoE model are exclusive to individual experts. Many are shared across the model, contributing to its efficiency. For instance, in a MoE model like Mixtral 8x7B, the dense equivalent parameter count might be less than the sum total of all experts due to shared components.
- Inference Speed: The inference speed benefits stem from the model only engaging a subset of experts for each token, effectively reducing the computational load to that of a much smaller model, while maintaining the benefits of a large parameter space.
Mamba is an innovative recurrent neural network architecture that stands out for its efficiency in handling long sequences, potentially up to 1 million elements. This model has garnered attention for being a strong competitor to the well-known Transformer models due to its impressive scalability and faster processing capabilities. Here's a simplified overview of what Mamba is and why it's significant:
Core Features of Mamba:
- Linear Time Processing: Unlike Transformers, which suffer from computational and memory costs that scale quadratically with sequence length, Mamba operates in linear time. This makes it much more efficient, especially for very long sequences.
- Selective State Spaces: Mamba employs selective state spaces, allowing it to manage and process lengthy sequences effectively by focusing on relevant parts of the data at any given time.
Selective State Spaces (SSS) in the context of models like Mamba refer to a sophisticated approach in neural network architecture that enables the model to efficiently handle and process very long sequences of data. This approach is particularly designed to improve upon the limitations of traditional models like Transformers and Recurrent Neural Networks (RNNs) when dealing with sequences of significant length. Here’s a breakdown of the key concepts behind Selective State Spaces:
Basis of Selective State Spaces:
- State Space Models (SSMs): At the core, SSS builds upon the concept of State Space Models. SSMs are a class of models used for describing systems that evolve over time, capturing dynamics through state variables that change in response to external inputs. SSMs have been used in various fields, such as signal processing, control systems, and now, in sequence modeling for AI.
- Selectivity Mechanism: The "selective" aspect introduces a mechanism that allows the model to determine which parts of the input sequence are relevant at any given time. This is achieved through a gating or routing function that dynamically selects which state space (or subset of the model's parameters) should be activated based on the input. This selective activation helps the model to focus its computational resources on the most pertinent parts of the data, enhancing efficiency.
Advantages Over Traditional Models:
- Efficiency with Long Sequences: Mamba's architecture is optimized for speed, offering up to five times faster throughput than Transformers while handling long sequences more effectively.
- Versatility: While its prowess is evident in text-based applications like chatbots and summarization, Mamba also shows potential in other areas requiring the analysis of long sequences, such as audio generation, genomics, and time series data.
- Innovative Design: The model builds on state space models (S4) but introduces a novel approach by incorporating selective structured state space sequence models, which enhance its processing capabilities.
Mamba represents a significant advancement in sequence modeling, offering a more efficient alternative to Transformers for tasks involving long sequences. Its ability to scale linearly with sequence length without a corresponding increase in computational and memory requirements makes it a promising tool for a wide range of applications beyond just natural language processing.
In essence, Mamba is redefining what's possible in AI sequence modeling, combining the best of RNNs and state space models with innovative techniques to achieve high efficiency and performance across various domains.
The RWKV architecture represents a novel approach in the realm of neural network models, integrating the strengths of Recurrent Neural Networks (RNNs) with the transformative capabilities of transformers. This hybrid architecture, spearheaded by Bo Peng and supported by a vibrant community, aims to address specific challenges in processing long sequences of data, making it particularly intriguing for various applications in Natural Language Processing (NLP) and beyond.
Key Features of RWKV:
- Efficiency in Handling Long Sequences: Unlike traditional transformers that struggle with quadratic computational and memory costs as sequence lengths increase, RWKV is designed to scale linearly. This makes it adept at efficiently processing sequences that are significantly longer than those manageable by conventional models.
- RNN and Transformer Hybrid: RWKV combines RNNs' ability to handle sequential data with the transformer's powerful self-attention mechanism. This fusion aims to leverage the best of both worlds: the sequential data processing capability of RNNs and the context-aware, parallel processing strengths of transformers.
- Innovative Architecture: RWKV introduces a simplified and optimized design that allows it to operate effectively as an RNN. It incorporates additional features such as TokenShift and SmallInitEmb to enhance performance, enabling it to achieve results comparable to those of GPT models.
- Scalability and Performance: With the infrastructure to support training models up to 14B parameters and optimizations to overcome issues like numerical instability, RWKV presents a scalable and robust framework for developing advanced AI models.
Advantages over Traditional Models:
- Handling Very Long Contexts: RWKV can utilize contexts of thousands of tokens and beyond, surpassing traditional RNN limitations and enabling more comprehensive understanding and generation of text.
- Parallelized Training: Unlike conventional RNNs that are challenging to parallelize, RWKV's architecture allows for faster training, akin to "linearized GPT," providing both speed and efficiency.
- Memory and Speed Efficiency: RWKV models can be trained and run with long contexts without the significant RAM requirements of large transformers, offering a balance between computational resource use and model performance.
Applications and Integration:
RWKV's architecture makes it suitable for a wide range of applications, from pure language models to multi-modal tasks. Its integration into the Hugging Face Transformers library facilitates easy access and utilization by the AI community, supporting a variety of tasks including text generation, chatbots, and more.
In summary, RWKV represents an exciting development in AI research, combining RNNs' sequential processing advantages with the contextual awareness and efficiency of transformers. Its design addresses key challenges in long sequence modeling, offering a promising tool for advancing NLP and related fields.
- LLM Agents: https://www.promptingguide.ai/research/llm-agents
- LLM Powered Autonomous Agents: https://lilianweng.github.io/posts/2023-06-23-agent/
- Emerging Trends in LLM Architecture- https://medium.com/@bijit211987/emerging-trends-in-llm-architecture-a8897d9d987b
- Four LLM trends since ChatGPT and their implications for AI builders: https://towardsdatascience.com/four-llm-trends-since-chatgpt-and-their-implications-for-ai-builders-a140329fc0d2