eval script add tool choice

[sfeng33]

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Summary by CodeRabbit

• New Features
  • Added optional tool choice parameter to evaluation runs, providing enhanced control over tool selection behavior. The parameter seamlessly integrates into the evaluation pipeline while maintaining full backward compatibility—existing configurations remain unaffected when not supplied.

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[coderabbitai]

📝 Walkthrough

Walkthrough

Introduces an optional TOOL_CHOICE parameter to the BFCL evaluation script that flows from the shell script through to the Python entrypoint. When provided, the parameter is injected into API requests via runtime monkey-patching of the generate_with_backoff function. Default behavior remains unchanged when the parameter is omitted.

Changes

Cohort / File(s)	Summary
BFCL Tool Choice Parameter .buildkite/scripts/tool_call/run-bfcl-eval.sh, ...bfcl-eval (Python entrypoint)	Added BFCL_TOOL_CHOICE parameter to shell script; displays value in output header; passes through CLI arguments to Python invocation. Python entrypoint reads tool_choice from argv and conditionally applies runtime monkey-patch to inject tool_choice into API requests via generate_with_backoff interception.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A choice has come, both swift and bright,
Through scripts it flows, from left to right,
The monkey patches in the night,
Make API calls feel just right! ✨
One parameter, so lean and light! 🎯

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'eval script add tool choice' clearly relates to the main change: adding a TOOL_CHOICE parameter to the BFCL evaluation script.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch bfcl-tool-choice

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can approve the review once all CodeRabbit's comments are resolved.

Enable the reviews.request_changes_workflow setting to automatically approve the review once all CodeRabbit's comments are resolved.