feat: Add --quiet/-q option to ask command to suppress verbose output - #847
dotanalter wants to merge 6 commits into
Conversation
|
Dotan Alter seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
WalkthroughAdds a global quiet flag threaded through CLI, config, interactive loop, LLM tool-calling, tool invocation, and toolset management to suppress non-essential logs and UI output; behavior unchanged when quiet is False. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI as holmes/main.py
participant Config as Config
participant TSM as ToolsetManager
participant LLM as ToolCallingLLM
participant Tool as Tool
User->>CLI: holmes ask ... --quiet/-q
CLI->>Config: create_console_toolcalling_llm(..., quiet)
Config->>TSM: list_console_toolsets(..., quiet)
TSM->>TSM: load/refresh/check (quiet: suppress logs)
CLI->>LLM: call(..., quiet)
LLM->>Tool: invoke(..., quiet)
Tool-->>LLM: result
LLM-->>CLI: LLMResult
CLI-->>User: AI output only (quiet)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (6)
🚧 Files skipped from review as they are similar to previous changes (6)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🔭 Outside diff range comments (2)
holmes/interactive.py (1)
1055-1056: Trace URL should be suppressed in quiet mode (currently printed unconditionally)The PR objective specifies suppressing trace URLs in quiet mode. This block should gate on
not quiet.Apply:
- if trace_url: - console.print(f"🔍 View trace: {trace_url}") + if trace_url and not quiet: + console.print(f"🔍 View trace: {trace_url}")holmes/core/tool_calling_llm.py (1)
185-193: Fix mutable default in LLMResult.instructionsinstructions uses a mutable default list which is shared across instances. Use Field(default_factory=list).
Apply these diffs:
@@ -from pydantic import BaseModel +from pydantic import BaseModel, Field @@ class LLMResult(BaseModel): @@ - instructions: List[str] = [] + instructions: List[str] = Field(default_factory=list)Also applies to: 12-12
🧹 Nitpick comments (5)
holmes/core/tools.py (1)
225-229: Optional: Demote or gate 'Applying additional instructions' log to respect quiet modeWhen
additional_instructionsare used, an info log is still emitted unconditionally. This can surface in quiet mode, which aims to minimize non-essential logs.Consider demoting to debug or gating with quiet (requires a larger refactor to pass quiet through to
_invoke). Minimal change:- logging.info( - f"Applying additional instructions: {self.additional_instructions}" - ) + logging.debug( + f"Applying additional instructions: {self.additional_instructions}" + )holmes/interactive.py (1)
1013-1013: Optional: Consider suppressing the “Thinking...” line in quiet modeNot required per the PR text, but for stricter quiet semantics, consider gating this cosmetic line.
- console.print(f"\n[bold {AI_COLOR}]Thinking...[/bold {AI_COLOR}]\n") + if not quiet: + console.print(f"\n[bold {AI_COLOR}]Thinking...[/bold {AI_COLOR}]\n")holmes/main.py (2)
238-241: Optional: Suppress the “Interactive mode disabled when reading piped input” message in quiet modeFor fully minimal output, consider gating this info message by
not quiet:- if interactive: - console.print( - "[bold yellow]Interactive mode disabled when reading piped input[/bold yellow]" - ) + if interactive: + if not quiet: + console.print( + "[bold yellow]Interactive mode disabled when reading piped input[/bold yellow]" + ) interactive = False
273-275: Optional: Suppress “Loaded prompt from file …” in quiet modeThis is informative but non-essential. Suppressing in quiet mode keeps output minimal:
- console.print( - f"[bold yellow]Loaded prompt from file {prompt_file}[/bold yellow]" - ) + if not quiet: + console.print( + f"[bold yellow]Loaded prompt from file {prompt_file}[/bold yellow]" + )holmes/core/tool_calling_llm.py (1)
246-256: Respect quiet for the post-processing announcementWhen quiet=True, suppress “Running post processing on investigation.” to avoid extra noise in quiet mode.
Apply this diff:
@@ - if post_process_prompt and user_prompt: - logging.info("Running post processing on investigation.") + if post_process_prompt and user_prompt: + if not quiet: + logging.info("Running post processing on investigation.") raw_response = text_response post_processed_response = self._post_processing_call( prompt=user_prompt, investigation=raw_response, user_prompt=post_process_prompt, )Also applies to: 347-355
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these settings in your CodeRabbit configuration.
📒 Files selected for processing (6)
holmes/config.py(3 hunks)holmes/core/tool_calling_llm.py(9 hunks)holmes/core/tools.py(4 hunks)holmes/core/toolset_manager.py(11 hunks)holmes/interactive.py(3 hunks)holmes/main.py(7 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Use Ruff for formatting and linting (configured in pyproject.toml)
Type hints required (mypy configuration in pyproject.toml)
Pre-commit hooks enforce quality checks
Don't add convenience logs that give away the problem
Don't write logs that directly state the issue
Ensure historical timestamps are properly handled in logs (especially with Loki)
Files:
holmes/core/tools.pyholmes/config.pyholmes/interactive.pyholmes/core/toolset_manager.pyholmes/core/tool_calling_llm.pyholmes/main.py
🧬 Code Graph Analysis (5)
holmes/config.py (2)
tests/plugins/toolsets/test_prometheus_integration.py (1)
tool_executor(21-28)holmes/main.py (1)
refresh_toolsets(957-967)
holmes/interactive.py (1)
tests/core/test_prompt.py (1)
console(13-14)
holmes/core/toolset_manager.py (1)
holmes/core/tools.py (2)
Toolset(335-479)check_prerequisites(411-468)
holmes/core/tool_calling_llm.py (1)
holmes/core/tools.py (1)
invoke(142-164)
holmes/main.py (3)
holmes/core/tool_calling_llm.py (1)
call(246-407)holmes/utils/console/result.py (1)
handle_result(12-38)tests/core/test_prompt.py (1)
console(13-14)
🔇 Additional comments (23)
holmes/config.py (2)
255-269: Quiet flag correctly threaded into console ToolExecutor creation
quietis added to the signature and forwarded tolist_console_toolsets(...), ensuring toolset loading status logs are suppressed when requested. Default remains backward compatible.
294-301: Quiet propagation into console ToolCallingLLM path is correctThe
quietflag is accepted bycreate_console_toolcalling_llm(...)and correctly passed tocreate_console_tool_executor(...). Since quiet behavior is runtime (affecting printing/logging), not constructor time, not passing it toToolCallingLLM's constructor is appropriate.holmes/core/tools.py (2)
142-150: Tool invocation logs are now quiet-awareGating the “Running tool …” and “Finished …” logs on
not quietaligns with CLI quiet mode and preserves defaults. No functional behavior changed.Also applies to: 160-164
411-469: Prerequisite checks respect quiet modeAdding
quiettoToolset.check_prerequisitesand gating logs reduces noise as intended. Early return on failure remains unchanged and safe.holmes/interactive.py (3)
800-802: Interactive loop signature updated with quiet modeSignature addition is correct and backward compatible. Quiet threading is ready for downstream usage.
905-911: Welcome banner and initial user echo suppressed in quiet modeThis matches the PR’s quiet-mode requirements for interactive. Good call keeping defaults unchanged.
1026-1027: Quiet flag propagated to AI callEnsures that tool execution announcements and related logs are suppressed during interactive calls. Correct integration.
holmes/main.py (6)
218-223: CLI option --quiet/-q added with clear help textFlag definition is correct and backward compatible. Short alias matches conventions.
257-263: Quiet correctly passed into console LLM creation
quietis threaded down tocreate_console_toolcalling_llm, ensuring toolset loading honors quiet mode from the top level.
290-292: Non-interactive user echo gated by quietMeets the requirement to suppress user input echo when scripting with
--no-interactive.
294-305: Interactive path now receives quiet flagEnsures interactive loop honors quiet flags (e.g., banner suppression). Correct forwarding.
321-321: Quiet propagated to ai.call(...) in non-interactive flowThis enables suppression of tool announcements and related logs during standard ask flow.
349-351: Trace URL output is quiet-awareMatches the PR statement to suppress trace URLs in quiet mode.
holmes/core/toolset_manager.py (4)
68-76: Quiet mode threaded through listing and prerequisites – correct and safe
_list_all_toolsets(..., quiet=False)added with correct default.- Prerequisite checks are invoked concurrently and are now quiet-aware via
check_toolset_prerequisites(..., quiet=quiet).- Backward compatibility preserved.
Also applies to: 122-135, 136-143
183-205: Quiet-aware refresh and caching logs
refresh_toolset_statusreceivesquietand passes it forward. The caching info log is properly gated bynot quiet. Behavior remains unchanged when quiet=False.Also applies to: 221-223
224-231: Quiet respected when loading with status and using cache
load_toolset_with_statuspicks upquiet.- Refresh announcements and “Using N datasources” logs are gated.
- Both cached and CLI toolset prereq checks invoke
check_toolset_prerequisites(..., quiet=quiet).Also applies to: 239-246, 258-259, 281-282, 301-306
310-325: Console toolset listing accepts quiet and propagates itThis closes the loop for CLI paths using quiet mode. Looks good.
holmes/core/tool_calling_llm.py (6)
373-379: Good: suppressed “AI requested N tool call(s)” when quietThis matches the PR objective to trim tool-call chatter in quiet mode.
391-392: Good: quiet propagated to each tool invocationPassing quiet down to _invoke_tool ensures tools can suppress “Running/Finished” logs (as tools.invoke honors quiet).
403-406: Good: suppressed extra blank line when quietAvoids stray whitespace in quiet output.
409-416: Good: _invoke_tool receives quiet and forwards to tool.invokeThis completes the quiet propagation chain to the tool layer.
Also applies to: 476-476
239-243: Addquietto prompt_call and IssueInvestigator.investigate and forward itConfirmed:
prompt_calldoes NOT accept/forwardquiet, andIssueInvestigator.investigatecallsprompt_callwithout any way to forwardquiet. ripgrep found these call sites that will be affected and need to either passquietor accept the new default:
- Change locations to update:
- holmes/core/tool_calling_llm.py — add
quiet: bool = Falsetoprompt_call(around line ~211) and forwardquietintoself.call.- holmes/core/tool_calling_llm.py — add
quiet: bool = FalsetoIssueInvestigator.investigate(around line ~822) and pass it intoprompt_call.- Call sites that currently call
prompt_call(and so may need to passquietwhere appropriate):
- server.py:222 (ai.prompt_call(...))
- holmes/main.py:669 (ai.prompt_call(...))
- examples/custom_llm.py:60 (ai.prompt_call(...))
- Note: my search did not find any direct uses of ToolCallingLLM.call/messages_call/call_stream to update.
Apply these diffs:
*** prompt_call signature and call *** @@ def prompt_call( self, system_prompt: str, user_prompt: str, post_process_prompt: Optional[str] = None, response_format: Optional[Union[dict, Type[BaseModel]]] = None, sections: Optional[InputSectionsDataType] = None, trace_span=DummySpan(), - ) -> LLMResult: + quiet: bool = False, + ) -> LLMResult: @@ - return self.call( + return self.call( messages, post_process_prompt, response_format, user_prompt=user_prompt, sections=sections, - trace_span=trace_span, + trace_span=trace_span, + quiet=quiet, )*** IssueInvestigator.investigate signature and prompt_call invocation *** @@ def investigate( self, issue: Issue, prompt: str, instructions: Optional[ResourceInstructions], console: Optional[Console] = None, global_instructions: Optional[Instructions] = None, post_processing_prompt: Optional[str] = None, sections: Optional[InputSectionsDataType] = None, trace_span=DummySpan(), - ) -> LLMResult: + quiet: bool = False, + ) -> LLMResult: @@ - res = self.prompt_call( + res = self.prompt_call( system_prompt, user_prompt, post_processing_prompt, response_format=response_format, sections=sections, trace_span=trace_span, + quiet=quiet, )Likely an incorrect or invalid review comment.
576-585: Streaming: quiet is respected; no server-side console prints for START_TOOL/TOOL_RESULT/ANSWER_ENDVerified: call_stream forwards quiet to tool invocations and the stream formatters only emit SSE events — there are no repo-side console handlers subscribing to StreamEvents that would print start/finish announcements.
- holmes/core/tool_calling_llm.py
- call_stream forwards quiet into executor.submit (lines ~682-689).
- _invoke_tool forwards quiet to tool.invoke (line ~476).
- Non-streaming call() already conditions console announcements on not quiet (lines ~373-406).
- holmes/utils/stream.py
- StreamEvents/StreamMessage and stream_investigate_formatter / stream_chat_formatter produce SSE via create_sse_message and do not print to stdout (lines ~8-22, 25-43, 46-61).
- server.py
- Streaming endpoints use those formatters to emit SSE to clients (/api/stream/investigate lines ~162-173; /api/chat streaming lines ~346-352).
No action required; changes can be approved.
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (2)
holmes/main.py (2)
238-241: Honor quiet for the “Interactive mode disabled when reading piped input” noticeThis is informational and should be suppressed in quiet mode to truly “only print the AI output.”
Apply:
- if interactive: - console.print( - "[bold yellow]Interactive mode disabled when reading piped input[/bold yellow]" - ) - interactive = False + if interactive: + if not quiet: + console.print( + "[bold yellow]Interactive mode disabled when reading piped input[/bold yellow]" + ) + interactive = False
273-275: Suppress “Loaded prompt from file …” in quiet modeThis informational message should also be gated by quiet to avoid extra output in scripts.
Apply:
- console.print( - f"[bold yellow]Loaded prompt from file {prompt_file}[/bold yellow]" - ) + if not quiet: + console.print( + f"[bold yellow]Loaded prompt from file {prompt_file}[/bold yellow]" + )
🧹 Nitpick comments (1)
holmes/main.py (1)
218-223: Add --quiet/-q flag: good; refine help text and ensure precedence vs --verbose is explicitNice addition. Two small improvements:
- Clarify that errors are still printed in quiet mode.
- When both -q and -v are passed, make it explicit that quiet suppresses verbosity. Today, logs configured by init_logging may still emit INFO/WARN if verbosity was requested.
Suggested help text tweak:
- quiet: bool = typer.Option( - False, - "--quiet", - "-q", - help="Quiet mode: only print the AI output", - ), + quiet: bool = typer.Option( + False, + "--quiet", + "-q", + help="Quiet mode: suppress non-essential logs and only print the AI output (errors still shown)", + ),Optionally ensure quiet wins over verbose logs (errors still shown). One minimal approach after init_logging:
# Right after init_logging(...) if quiet: logging.getLogger().setLevel(logging.ERROR)If you prefer a cleaner API, consider adding a quiet parameter to init_logging and set the handler level accordingly. I can follow up with a small PR to wire this end-to-end and add CLI tests for quiet. Do you want that?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these settings in your CodeRabbit configuration.
📒 Files selected for processing (1)
holmes/main.py(6 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Use Ruff for formatting and linting (configured in pyproject.toml)
Type hints required (mypy configuration in pyproject.toml)
Pre-commit hooks enforce quality checks
Don't add convenience logs that give away the problem
Don't write logs that directly state the issue
Ensure historical timestamps are properly handled in logs (especially with Loki)
Files:
holmes/main.py
🧬 Code Graph Analysis (1)
holmes/main.py (1)
holmes/core/tool_calling_llm.py (1)
call(246-407)
🔇 Additional comments (5)
holmes/main.py (5)
290-291: Suppressing user prompt echo when quiet — LGTMCorrect gating: avoids echoing the user prompt in non-interactive quiet runs while preserving default behavior otherwise.
293-305: Interactive loop receives quiet — LGTMPassing quiet into run_interactive_loop is consistent with the intended UX (suppress banner/prompt noise while keeping the interaction flow).
321-322: Forwarding quiet into ai.call — LGTMThis aligns with the tool-calling suppression implemented in tool_calling_llm and ensures minimal noise in the ask path.
349-351: Hiding trace URL when quiet — LGTMMatches the design goal to keep only the AI output when quiet.
257-262: Quiet propagated to LLM factory — signatures verifiedVerified that forwarding quiet is safe: the upstream callsites accept a quiet parameter, so no runtime TypeError expected.
- holmes/config.py: create_console_toolcalling_llm (line 289) — params=['self', 'dal', 'refresh_toolsets', 'tracer', 'quiet']
- holmes/interactive.py: run_interactive_loop (line 790) — params=[..., 'quiet', ...]
- holmes/core/tool_calling_llm.py: call (line 246) — params=[..., 'quiet']
No changes required.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
holmes/core/tool_calling_llm.py (2)
349-356: Suppress post-processing status line when quiet is enabledThis informational log is non-essential output. It should be gated by quiet for consistency with the PR objective to suppress status/announcement logs.
Apply this diff:
- logging.info("Running post processing on investigation.") + if not quiet: + logging.info("Running post processing on investigation.") raw_response = text_response post_processed_response = self._post_processing_call( prompt=user_prompt, investigation=raw_response, user_prompt=post_process_prompt, )
211-231: Quiet isn’t propagated via prompt_call()/IssueInvestigator.investigate() — potential gapIf any path uses prompt_call() (e.g., IssueInvestigator.investigate()), quiet won’t be honored because prompt_call lacks the parameter and doesn’t forward it. Consider adding quiet to these APIs and passing it through.
Apply these diffs to wire quiet end-to-end:
Prompt path:
def prompt_call( self, system_prompt: str, user_prompt: str, post_process_prompt: Optional[str] = None, response_format: Optional[Union[dict, Type[BaseModel]]] = None, sections: Optional[InputSectionsDataType] = None, - trace_span=DummySpan(), + trace_span=DummySpan(), + quiet: bool = False, ) -> LLMResult: @@ return self.call( messages, post_process_prompt, response_format, user_prompt=user_prompt, sections=sections, trace_span=trace_span, + quiet=quiet, )Investigation path:
def investigate( self, issue: Issue, prompt: str, instructions: Optional[ResourceInstructions], console: Optional[Console] = None, global_instructions: Optional[Instructions] = None, post_processing_prompt: Optional[str] = None, sections: Optional[InputSectionsDataType] = None, - trace_span=DummySpan(), + trace_span=DummySpan(), + quiet: bool = False, ) -> LLMResult: @@ res = self.prompt_call( system_prompt, user_prompt, post_processing_prompt, response_format=response_format, sections=sections, - trace_span=trace_span, + trace_span=trace_span, + quiet=quiet, )If the CLI/interactive layer never uses these paths when quiet is set, feel free to skip — but it’s a common surface that could otherwise leak non-quiet logs.
Also applies to: 739-836
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
holmes/core/tool_calling_llm.py(9 hunks)holmes/core/tools.py(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- holmes/core/tools.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Type hints are required (project is type-checked with mypy)
Use Ruff for formatting and linting (configured in pyproject.toml)
Files:
holmes/core/tool_calling_llm.py
🧬 Code Graph Analysis (1)
holmes/core/tool_calling_llm.py (2)
holmes/core/tools_utils/tool_executor.py (1)
invoke(49-58)holmes/core/tools.py (1)
invoke(148-171)
🔇 Additional comments (9)
holmes/core/tool_calling_llm.py (9)
239-243: messages_call: Quiet flag pass-through looks correctThreading quiet into call(...) preserves existing callers and behavior.
255-256: call signature: Quiet flag addition is safe and backwards-compatibleAppended at the end, so positional callers remain unaffected.
377-381: Good: Suppresses “AI requested N tool call(s)” when quietMatches the desired behavior to remove tool execution announcements.
393-394: Good: Quiet is forwarded to _invoke_tool in non-streaming pathEnsures tools respect quiet during execution.
406-407: Good: Blank line after batch completion suppressed in quiet modeAvoids extra whitespace in quiet output.
417-418: _invoke_tool signature updated with quiet — consistent with call sitesThis aligns the signature with new usage and keeps propagation explicit.
478-479: Good: Quiet is passed to Tool.invoke(...)This matches tools.py’s quiet-aware logging and fulfills the suppression requirement for per-tool “Running/Finished” logs.
585-586: call_stream: Quiet flag added — verify downstream handling of stream eventsYou still yield START_TOOL and TOOL_RESULT events in streaming mode. That’s fine if upper layers (interactive/CLI) suppress printing based on quiet. Please confirm that consumers of StreamMessage honor quiet to avoid reintroducing announcements.
Do you want me to scan the repo to confirm that StreamEvents are conditionally printed based on quiet?
692-693: Good: Quiet forwarded to _invoke_tool in streaming pathTool execution logs should be suppressed consistently here too.
|
@dotanalter can you please sign the CLA? #847 (comment) |
|
I'm going to close this PR because there is an option to output into json file and take only result which is the best option for me. |
3 participants
🎯 Summary
Adds a new
--quiet/-qoption to theholmes askcommand that suppresses verbose output while preserving essential AI responses. This feature is useful for scripting, automation, and users who prefer minimal output.🔧 Changes Made
Core Implementation:
--quiet/-qCLI option to theaskcommand with Typer integrationmain.py→config.py→toolset_manager.py→tools.py/tool_calling_llm.pyOutput Suppression:
Preserved Functionality:
--no-interactive📋 Usage Examples
🧪 Testing
The implementation has been manually tested and works correctly in both interactive and non-interactive modes. The quiet parameter flows properly through all components without affecting core functionality.
🔄 Backwards Compatibility
This enhancement addresses user requests for cleaner output when using Holmes in automated workflows while maintaining the rich interactive experience for manual use.