feat: add /compare-models and /compare-models-free slash commands (t168)#660
feat: add /compare-models and /compare-models-free slash commands (t168)#660marcusquinn merged 1 commit intomainfrom
Conversation
… model capability comparison (t168) Create subagent, helper script, and two slash commands for comparing AI model capabilities, pricing, context windows, and task suitability across providers. Components: - .agents/tools/ai-assistants/compare-models.md: Subagent with full comparison workflow - .agents/scripts/compare-models-helper.sh: CLI with embedded model database (17 models, 6 providers) supporting list/compare/recommend/pricing/context/capabilities/providers - .agents/scripts/commands/compare-models.md: Full comparison with optional live web fetch - .agents/scripts/commands/compare-models-free.md: Offline comparison using embedded data only - .agents/subagent-index.toon: Updated with new script and subagent reference Design decisions: - Embedded model data in helper script for offline/free usage (no API calls needed) - Partial name matching for model queries (e.g. 'sonnet' matches 'claude-sonnet-4') - Task recommendation engine maps 15 task types to best/runner-up/budget models - Maps to aidevops internal tiers (haiku/flash/sonnet/pro/opus) - Used heredoc redirect instead of pipe for while loops to avoid subshell variable scoping
Summary of ChangesHello @marcusquinn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the AI model management capabilities by introducing a robust system for comparing various AI models. It provides users with flexible tools to evaluate models based on pricing, context windows, and capabilities, either using an embedded, up-to-date database for offline use or by fetching live data for the most current information. This empowers users to make informed decisions when selecting AI models for specific tasks, improving efficiency and cost-effectiveness within the AI DevOps framework. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Caution Review failedThe pull request is closed. WalkthroughIntroduces a comprehensive AI model comparison framework with offline-capable documentation files, a Bash helper script featuring embedded model data and CLI commands for listing/comparing/recommending models, and metadata updates. Enables model evaluation across capabilities, pricing, context windows, and task suitability without external API calls. Changes
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly Related Issues
Poem
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔍 Code Quality Report�[0;35m[MONITOR]�[0m Code Review Monitoring Report �[0;34m[INFO]�[0m Latest Quality Status: �[0;34m[INFO]�[0m Recent monitoring activity: 📈 Current Quality Metrics
Generated on: Sun Feb 8 20:49:13 UTC 2026 Generated by AI DevOps Framework Code Review Monitoring |
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new feature to compare AI models through two slash commands, /compare-models and /compare-models-free, supported by a new helper script compare-models-helper.sh. The implementation is well-structured and the documentation is clear. My review focuses on the new bash script, where I've identified a high-severity security issue related to potential code injection in awk commands. I've provided a suggestion to refactor this for improved security and readability, in line with the repository's style guide on safe command construction.
| # Use awk for float comparison | ||
| if awk "BEGIN{exit !($input < $cheapest_input_price)}"; then | ||
| cheapest_input="$model_id" | ||
| cheapest_input_price="$input" | ||
| fi | ||
| if awk "BEGIN{exit !($output < $cheapest_output_price)}"; then | ||
| cheapest_output="$model_id" | ||
| cheapest_output_price="$output" | ||
| fi |
There was a problem hiding this comment.
Directly embedding shell variables into the awk script string poses a potential code injection vulnerability and reduces readability. It's much safer to pass variables to awk using the -v flag. This change refactors the float comparison to use this safer method.
| # Use awk for float comparison | |
| if awk "BEGIN{exit !($input < $cheapest_input_price)}"; then | |
| cheapest_input="$model_id" | |
| cheapest_input_price="$input" | |
| fi | |
| if awk "BEGIN{exit !($output < $cheapest_output_price)}"; then | |
| cheapest_output="$model_id" | |
| cheapest_output_price="$output" | |
| fi | |
| # Use awk for float comparison, passing variables safely with -v | |
| if awk -v price="$input" -v cheapest="$cheapest_input_price" 'BEGIN{exit !(price < cheapest)}'; then | |
| cheapest_input="$model_id" | |
| cheapest_input_price="$input" | |
| fi | |
| if awk -v price="$output" -v cheapest="$cheapest_output_price" 'BEGIN{exit !(price < cheapest)}'; then | |
| cheapest_output="$model_id" | |
| cheapest_output_price="$output" | |
| fi |
References
- The style guide (line 32) prohibits using
evaland recommends safe ways to construct dynamic commands. While not strictlyeval, injecting shell variables directly into anawkscript string is a similar code injection risk that should be avoided. (link)
1 participant
Summary
/compare-modelsand/compare-models-freeslash commands for AI model capability comparison across providerscompare-models-helper.shCLI with embedded database of 17 models from 6 providers (Anthropic, OpenAI, Google, DeepSeek, Meta)compare-models.mdsubagent intools/ai-assistants/with full comparison workflowWhat's New
Slash Commands
/compare-models [models...] [--task TASK]— Full comparison with optional live web fetch for latest pricing/compare-models-free [models...] [--task TASK]— Offline comparison using embedded data only (no API calls, no web fetches)Helper Script (
compare-models-helper.sh)7 subcommands:
list,compare,recommend,pricing,context,capabilities,providersFeatures:
sonnetmatchesclaude-sonnet-4)Subagent
tools/ai-assistants/compare-models.md— Orchestrates the comparison workflow, optionally enriches with live pricing dataFiles Changed
.agents/tools/ai-assistants/compare-models.md.agents/scripts/compare-models-helper.sh.agents/scripts/commands/compare-models.md.agents/scripts/commands/compare-models-free.md.agents/subagent-index.toonQuality
local var="$1", explicit returns,shared-constants.shsourcingSummary by CodeRabbit
Documentation
New Features