Skip to content

Added a new toolset to HolmesGPT - #7

Closed
adithyapaib wants to merge 2 commits into
HolmesGPT:masterfrom
adithyapaib:adithyapaib
Closed

adithyapaib wants to merge 2 commits into
HolmesGPT:masterfrom
adithyapaib:adithyapaib

Conversation

@adithyapaib

Copy link
Copy Markdown

Added the following Toolsets

  1. Add a toolset to check what Jenkins jobs are running at the moment
  2. Add a toolset to query Jaeger traces

@CLAassistant

CLAassistant commented Jun 2, 2024 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@aantn

aantn commented Jun 10, 2024

Copy link
Copy Markdown
Collaborator

Hey, thank you for contributing!

There are two issues we need to resolve before merging:

  1. For Jaeger, querying by trace_id generally will not be useful, as HolmesGPT has no way of knowing the trace ID upfront and wont be able to craft an appropriate API call. It may be more useful here to take one of the following approaches

(a) Query Jaeger's query dependency graph (see here) so that you can ask questions like what services are speaking to myservice

(b) Allow querying a list of all Jaeger services and then traces for that service - see here for example

  1. For Jenkins, the added tool makes sense (you can ask about faiing jenkins jobs using it), but I want to make some changes to Holmes before merging so that can give {{ jenkins_url }} and similar parameters special treatment as secrets that the user defines, and not regular function parameters injected by the LLM. This part is on our side.

@aantn

aantn commented Jul 11, 2024

Copy link
Copy Markdown
Collaborator

Hey, I'm closing this for now but please feel free to re-open after addressing the comments above!

@aantn aantn closed this Jul 11, 2024
goyamegh added a commit to goyamegh/holmesgpt that referenced this pull request Mar 9, 2026
- Rename OTEL_AWS_SERVICE → HOLMES_AWS_OSIS_SERVICE with backwards-compat
  fallback (Comment HolmesGPT#10, svrnm)
- Align OTEL_DEBUG with OTEL spec OTEL_LOG_LEVEL=debug with backwards-compat
  fallback (Comment HolmesGPT#9, svrnm)
- Add ml-commons AgentTracer.java GitHub permalink (Comment HolmesGPT#3, kylehounslow)
- Document needs_aws_auth() as single source of truth with consumer list
  (Comment HolmesGPT#4, kylehounslow)
- Clarify otel_logging.py: logs NOT exported via OTLP, naming avoids
  shadowing builtin logging (Comment HolmesGPT#5, kylehounslow)
- Explain experimental/ placement: API evolving, removable via try/except
  no-op fallbacks (Comment HolmesGPT#6, kylehounslow)
- Clarify server.py middleware: tracing/metrics init in init_otel() above,
  middleware only handles per-request spans (Comment HolmesGPT#7, kylehounslow)
- Split env var docs into Standard OTEL / Holmes-Specific tables, add
  missing vars, add OSIS hyperlink (Comments HolmesGPT#1, HolmesGPT#9, HolmesGPT#10)
- Fix no-op tracer fallback in server-agui.py for when OTEL is unavailable

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Megha Goyal <goyamegh@amazon.com>
aantn pushed a commit that referenced this pull request Mar 21, 2026
Address 5 CodeRabbit review comments from PR #1811:

Issue #3 - Cache key inadequacy: create_tool_executor now stores a cache
key (tags, enable_all) alongside the executor and validates it on cache
hit. Callers with different parameters get a fresh executor.

Issue #4/#5 - Change detection misses added/removed toolsets:
refresh_toolsets_and_get_changes now compares name sets between old and
new toolset lists, reporting additions and removals alongside status
transitions. refresh_tool_executor always updates the cached executor
(not just when status changes exist).

Issue #6 - Race condition: Added threading.Lock around all reads and
writes of _cached_tool_executor. Added Config.cached_tool_executor
property for thread-safe external access. Updated server.py to use it.

Issue #7 - Cold-cache refresh: refresh_tool_executor now uses
FORCE_REFRESH on cold start instead of ENABLED, ensuring live
prerequisite checks rather than stale disk cache.

https://claude.ai/code/session_01Qr7wtBgYGnh1G6Bq2wPp79
Signed-off-by: Claude <noreply@anthropic.com>
@claude claude Bot mentioned this pull request Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants