feat: Add Runtime for tool calling #1163
Conversation
There was a problem hiding this comment.
seems we need to make the image public
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.45/images/create?tag=latest&fromImage=xukunliu%2Fcamel: Not Found ("pull access denied for xukunliu/camel, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")
There was a problem hiding this comment.
Hi Wendong! Just wondering if camel has any plans to build an official image and push it to DockerHub?
There was a problem hiding this comment.
Thanks @liuxukun2000 ! The core code looks great now, I added one more commit for small docstring and code format polish here: 4c4d4ac , feel free to check. The last thing is the unit test to be added, could you add unit test under test folder? Thanks a lot!
| @app.post(f"/{func.get_function_name()}") | ||
| async def dynamic_function(data: Dict, func=func): | ||
| redirect_stdout = data.get('redirect_stdout', False) | ||
| if redirect_stdout: | ||
| sys.stdout = io.StringIO() | ||
| response_data = func.func(*data['args'], **data['kwargs']) | ||
| if redirect_stdout: | ||
| sys.stdout.seek(0) | ||
| output = sys.stdout.read() | ||
| sys.stdout = sys.__stdout__ | ||
| return { | ||
| "output": json.dumps(response_data), | ||
| "stdout": output, | ||
| } | ||
| return {"output": json.dumps(response_data)} |
There was a problem hiding this comment.
Is RPC a better choice? We can have a simple discussion about it.
There was a problem hiding this comment.
Hi xukun, Would it be possible to incorporate the agent part into this example?
|
Hi, Xukun, As a framework, I think Camel shouldn't rely on starting an HTTP service for running tools. If the HTTP port is exposed, it introduces additional security concerns that need to be addressed. Would it be possible to directly start a Docker container, execute the corresponding tool script (.py), and destroy the container immediately after execution? |
|
Hi @Asher-hss , thank you for your suggestion and for considering the potential security implications of starting an HTTP service. However, I believe HTTP security concerns are not something the framework itself should manage, as long as the service is used properly and securely. HTTP servers are widely adopted for implementing function calls due to their versatility and ease of integration. |
|
@liuxukun2000 Hi xukun, I really appreciate the consideration for flexibility—it’s definitely an important factor. That said, I’m also thinking about starting a service inherently aims to create a long-running environment, which might not fully align with the original goals we had in mind for designing this runtime. Then, could you share an example scenario where maintaining the function call context would be particularly useful? Lastly, I’d love to hear your thoughts on what specific use cases you see for adopting an HTTP runtime. |
Description
Add runtime class for tool calling: #1014
Motivation and Context
Issue: #1014
Types of changes
What types of changes does your code introduce? Put an
xin all the boxes that apply:Checklist
Go over all the following points, and put an
xin all the boxes that apply.If you are unsure about any of these, don't hesitate to ask. We are here to help!