Adding first version of AgentEval -- a framework for assessing task utility for LLM-powered applications

[julianakiseleva]

• We introduce AgentEval — the first version of the framework to automatically assess task utility for an arbitrary application. It suggests criteria to explain task utility and then quantifies these criteria for logs of your system. AgentEval consists of two key components:

  • CriticAgent: This is an LLM-based agent that generates criteria to evaluate a given task.
  • QuantifierAgent: This agent quantifies the performance of any sample task based on the criteria designed by the CriticAgent.

• We demonstrate the usage of our framework with the Math Problems dataset in notebook that allows for running on single logs as well as plotting the quantified estimated performance for a set of problems.

• The model logs you can find them in agenteval branch

This PR has added:

• notebook/agenteval_cq_math.ipynb to demonstrate AgentEval using the math problems
• sample files that required to demonstrate the work of notebook in test/test_files/agenteval-in-out
• website/blog/2012-11-11-AgentEval -- the blog post to explain the AgentEval

Related issue number

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[Narabzad]

e.g., Previous Work is capitalized by other section titles use lower cas

@gagb done.