Making this library async

[rickardp]

Most if not all of the functionality of this library is network bound. Would it make sense to make its API asynchronous?

[taha-yassine]

I second this.
In my case I would like to implement some tools that are to be executed on a remote server and require the calls to be async.

[ericfeunekes]

Yes. This is critical if you want to run it with any sort of load. I want to put this on a server and even trying to put this threads could quickly cause issues.

I see there being two parts:

1. make the library code asynchronous
2. Prompt the LLM to write and execute asynchronous code.

The library is still small enough that it shouldn't be too difficult to write the API to be asynchronous. But it becomes harder the more features are built into it. Happy to contribute to this if there is interest and an idea of how to integrate/segregate it.

[rickardp]

1. make the library code asynchronous
2. Prompt the LLM to write and execute asynchronous code.

IMHO, even if 2) is hard to do, 1) adds value. The LLM part consumes a significant portion of the execution time and in order to contain the executed Python, some kind of cross process/sandbox solution would be needed anyway, so the Python execution would become a separate problem

[ericfeunekes]

IMHO, even if 2) is hard to do, 1) adds value.

Agreed. So the question is just the approach to take? You could mirror the base class with AsyncTool for exmample or just add a async_forward method. The models are easy because you just add:

async def __call_async__(self, *args, **kwargs):
    return await asyncio.to_thread(self.__call__, *args, **kwargs)

And then probably add an async method to MultiStepAgent.

Not sure how much time I have to do this, but would like to hear which approach the maintainers would like to take. I can try picking away at it.

[FAbrahamDev]

For what it's worth, I've forked the project and threw around some async and awaits which works for me.
Definitely not how it should be done 😅

https://github.com/respeak-io/smolagents_async

[freckletonj]

Just seconding this; some specific things you can't efficiently do serialized:

• get multiple llm answers and choose the best one
• process multiple web searches at once
• run multiple python envs at once

I for one would prefer if this library was orchestration only, and punted all heavy lifting to HTTP requests, eg to locally hosted services. IE VLLM can handle async requests beautifully, and a separate service could handle multiprocessing multiple python envs, and another service could do web search, etc. Then it'd free this library up to truly be a light weight agent lib, and critically, be async.

[skjortan23]

yes i like Smolagents but the lack of async make it really hard to chain agents. My use case would be spinning of multiple tools and awaiting the answers.

[alisher-amantay]

@FAbrahamDev you did an amazing job with supporting async in LocalPythonInterpreter, thanks