InMemoryRateLimiter does not work with BaseLLM child classes

[dylanbouchard]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The InMemoryRateLimiter is not used when passed to a child class of BaseLLM (e.g. VertexAI). It is only used for child classes of BaseChatModel (e.g. ChatVertexAI). Additionally, no exception is raised when passing InMemoryRateLimiter to BaseLLM.

from langchain_core.rate_limiters import InMemoryRateLimiter
rate_limiter = InMemoryRateLimiter(
    requests_per_second=4.5, 
    check_every_n_seconds=0.5, 
    max_bucket_size=280,  
)

from langchain_google_vertexai import ChatVertexAI, VertexAI
chat = ChatVertexAI(
    model_name="gemini-1.0-pro", rate_limiter=rate_limiter
) # Rate limiter will get used by this object
llm = VertexAI(
    model_name="gemini-1.0-pro", rate_limiter=rate_limiter
) # Rate limiter will not get used by this object

Error Message and Stack Trace (if applicable)

N/A

Description

The InMemoryRateLimiter is not used when passed to a child class of BaseLLM (e.g. VertexAI). It is only used for child classes of BaseChatModel (e.g. ChatVertexAI). Additionally, no exception is raised when passing InMemoryRateLimiter to BaseLLM. From what I can tell, this is not discussed anywhere in the documentation, so will likely cause confusion for many users.

From further investigation, I found that there is no use of InMemoryRateLimiter in the constructor of BaseLLM: https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/language_models/chat_models.py#L207

but there is in the constructor of BaseChatModel:
https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/language_models/llms.py#L292

System Info

System Information

OS: Linux
OS Version: #1 SMP Debian 5.10.226-1 (2024-10-03)
Python Version: 3.10.15 | packaged by conda-forge | (main, Sep 30 2024, 17:51:04) [GCC 13.3.0]

[keenborder786]

Yes that is intentional. RateLimiter are not supported with BaseLLM

[dylanbouchard]

Yes that is intentional. RateLimiter are not supported with BaseLLM

Understood. Are there plans to add this support? If not, it would help to clarify this in the documentation.

[EMIDY643]

I just came across an incredible tool that's incredibly useful! 🚀✨ If you want to give it a try, explore these resources for free access—no sign-up required! 🙌🔗
🔗 👉🔴 Unlock AI magic for free here! ✅

🔗 👉 🔴Explore free AI features now! ✅

3- Best undress ai for free without sing up :Click HERE >>
4- surprise For an iPhone at a surprising price :Click HERE >>
No cost: Completely free with no hidden charges.

          DeepNude AI offers advanced tools for generating high-quality AI-powered imagery. With features like precision, ease of use, and seamless integration, it caters to beginners and professionals alike. Users can enjoy: https://5454.us/

👉🔴 Click HERE >>

👉🔴 Click HERE >>

3- Best undress ai for free without sing up :https://5454.us/
4- surprise For an iPhone at a surprising price :https://installchecker.com/sl/7dr1n
No cost: Completely free with no hidden charges.

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

DeepNude AI offers advanced tools for generating high-quality AI-powered imagery. With features like precision, ease of use, and seamless integration, it caters to beginners and professionals alike. Users can enjoy: https://installchecker.com/sl/7dr1n

👉🔴 Click HERE >>

👉🔴 Click HERE >>

3- Best undress ai for free without sing up : https://5454.us/
4- surprise For an iPhone at a surprising price : https://installchecker.com/sl/7dr1n
No cost: Completely free with no hidden charges.

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

[👉🔴 Click HERE >>][(https://5454.us/?tdsId=s0331aes_r&tds_campaign=s0331aes&utm_sub=opnfnl&s1=ps&utm_source=int&subid={subid}&clickid={clickid}&subid2={subid2}&affid=181c7b69)

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

👉🔴 Click HERE >>

Accessibility: Available on various platforms.
Advanced results: Crisp and accurate outputs.
Why Choose DeepNude AI? https://installchecker.com/sl/7dr1n
Trusted worldwide for its efficiency, DeepNude AI combines simplicity with reliability, making it an ideal choice for users seeking fast, effective AI solutions

Originally posted by @comtthe in tidys/CocosCreatorPlugins#33 (comment)
#28439

[keenborder786]

@dylanbouchard not currently

[dylanbouchard]

@keenborder786 Can I create a PR to 1) update the documentation for rate limiter, 2) create a warning / exception when rate limiter is passed to a BaseLLM, and/or 3) make BaseLLM compatible with rate limiter? Please let me know what you think makes most sense.

As it is currently, this has caused a great deal of confusion for folks I work with.

[dosubot]

Hi, @dylanbouchard. I'm Dosu, and I'm helping the LangChain team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• You reported that InMemoryRateLimiter does not work with BaseLLM child classes like VertexAI.
• It functions correctly with BaseChatModel child classes such as ChatVertexAI.
• keenborder786 confirmed this behavior is intentional, and rate limiters are not supported with BaseLLM.
• You suggested documentation updates or warnings to clarify this behavior.
• keenborder786 mentioned no current plans to add rate limiter support to BaseLLM.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of LangChain. If so, you can keep the discussion open by commenting here.
• Otherwise, this issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!