Closed
Description
We have a few valuable debugging capabilities in AGS today
- Abilitity to test agent teams in teambuilder
- Automatic validation of entire team
However, validation only gives us static checks for agent behavior (e.g a spec is valid .. valid provider, etc), it tells us nothing about actual functional behaviour e.g., an agent might have the right provider, but have missing required fields eg a model client, or a model client look good but its baseulr incorrect etc.
This could be useful in the Gallery where developers can create components , test them and then reliably reuse them.
This PR is meant to add the ability to run a component as a form of live test.
Currently will focus on the ability to test model clients. Others can be tested relatively easily (e.g., agents) by using the test teams already available.
How
- Extend /validate endpoint to validate/test/component= component
- For each coponent determine how we will test
- agent - task.run("what is x")
- model - model.create ("hi there")
- tools - tool.run ...