Skip to content

Service Now - Adds health check table configuration - #1537

Merged
Sheeproid merged 2 commits into
masterfrom
servicenow-check
Feb 10, 2026
Merged

Sheeproid merged 2 commits into
masterfrom
servicenow-check

Conversation

@Sheeproid

@Sheeproid Sheeproid commented Feb 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Added optional api_key_header and health_check_table configuration fields for ServiceNow integration.
  • Improvements

    • Health checks now use the configured table name.
    • Improved diagnostics for connection failures and HTTP errors with more detailed messages.
  • Documentation

    • Updated ServiceNow docs to describe the new optional fields.
  • Tests

    • Updated tests to preserve and restore existing transformer registrations during setup/teardown.

Allows users to configure the table used for the ServiceNow health check.

This change enables users to specify a custom table for the health check to accommodate API keys with limited permissions, improving the tool's adaptability. It also includes error responses for debugging.

Signed-off-by: Tomer Keshet <tomer@robusta.dev>
@linux-foundation-easycla

linux-foundation-easycla Bot commented Feb 10, 2026 •

Copy link
Copy Markdown

CLA Not Signed

@Sheeproid Sheeproid changed the title Adds health check table configuration Service Now - Adds health check table configuration Feb 10, 2026
@netlify

netlify Bot commented Feb 10, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for holmes-docs ready!

Name Link
🔨 Latest commit e4f8307
🔍 Latest deploy log https://app.netlify.com/projects/holmes-docs/deploys/698b4c107a17060008f981b0
😎 Deploy Preview https://deploy-preview-1537--holmes-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@Sheeproid
Sheeproid requested a review from arikalon1 February 10, 2026 11:32
@github-actions

github-actions Bot commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

📂 Previous Runs

📜 Run @ 43511e9 (#21863182373)

✅ Results of HolmesGPT evals

Automatically triggered by commit 43511e9 on branch servicenow-check

View workflow logs

Results of HolmesGPT evals

  • ask_holmes: 9/9 test cases were successful, 0 regressions
Status Test case Time Turns Tools Cost
✅ 09_crashpod 31.1s 5 11 $0.2289
✅ 101_loki_historical_logs_pod_deleted 44.8s 6 11 $0.2640
✅ 111_pod_names_contain_service 38.5s 6 13 $0.2602
✅ 112_find_pvcs_by_uuid 34.6s 6 9 $0.2638
✅ 12_job_crashing 36.5s 5 12 $0.2448
✅ 176_network_policy_blocking_traffic_no_runbooks 45.2s 6 17 $0.2984
✅ 24_misconfigured_pvc 32.8s 5 13 $0.2301
✅ 43_current_datetime_from_prompt 4.8s 1 — $0.1051
✅ 61_exact_match_counting 15.7s 4 4 $0.1588
Total 31.6s avg 4.9 avg 11.2 avg $2.0541

✅ Results of HolmesGPT evals

Automatically triggered by commit e4f8307 on branch servicenow-check

View workflow logs

Results of HolmesGPT evals

  • ask_holmes: 9/9 test cases were successful, 0 regressions
Status Test case Time Turns Tools Cost
✅ 09_crashpod 30.7s 5 9 $0.2162
✅ 101_loki_historical_logs_pod_deleted 35.3s 5 9 $0.2327
✅ 111_pod_names_contain_service 32.9s 5 11 $0.2279
✅ 112_find_pvcs_by_uuid 32.5s 7 6 $0.2209
✅ 12_job_crashing 33.6s 6 11 $0.2329
✅ 176_network_policy_blocking_traffic_no_runbooks 53.6s 9 18 $0.3304
✅ 24_misconfigured_pvc 39.2s 7 16 $0.2572
✅ 43_current_datetime_from_prompt 4.9s 1 — $0.1048
✅ 61_exact_match_counting 12.5s 3 2 $0.1419
Total 30.6s avg 5.3 avg 10.2 avg $1.9650
📖 Legend
Icon Meaning
✅ The test was successful
➖ The test was skipped
⚠️ The test failed but is known to be flaky or known to fail
🚧 The test had a setup failure (not a code regression)
🔧 The test failed due to mock data issues (not a code regression)
🚫 The test was throttled by API rate limits/overload
❌ The test failed and should be fixed before merging the PR
🔄 Re-run evals manually

⚠️ Warning: /eval comments always run using the workflow from master, not from this PR branch. If you modified the GitHub Action (e.g., added secrets or env vars), those changes won't take effect.

To test workflow changes, use the GitHub CLI or Actions UI instead:

gh workflow run eval-regression.yaml --repo HolmesGPT/holmesgpt --ref servicenow-check -f markers=regression -f filter=

Option 1: Comment on this PR with /eval:

/eval
markers: regression

Or with more options (one per line):

/eval
model: gpt-4o
markers: regression
filter: 09_crashpod
iterations: 5

Run evals on a different branch (e.g., master) for comparison:

/eval
branch: master
markers: regression
Option Description
model Model(s) to test (default: same as automatic runs)
markers Pytest markers (no default - runs all tests!)
filter Pytest -k filter (use /list to see valid eval names)
iterations Number of runs, max 10
branch Run evals on a different branch (for cross-branch comparison)

Quick re-run: Use /rerun to re-run the most recent /eval on this PR with the same parameters.

Option 2: Trigger via GitHub Actions UI → "Run workflow"

🏷️ Valid markers

benchmark, chain-of-causation, compaction, confluence, context_window, coralogix, counting, database, datadog, datetime, easy, elasticsearch, embeds, fast, frontend, grafana-dashboard, hard, integration, kafka, kubernetes, leaked-information, logs, loki, medium, metrics, network, newrelic, no-cicd, numerical, one-test, port-forward, prometheus, question-answer, regression, runbooks, slackbot, storage, toolset-limitation, traces, transparency


Commands: /eval · /rerun · /list

CLI: gh workflow run eval-regression.yaml --repo HolmesGPT/holmesgpt --ref servicenow-check -f markers=regression -f filter=

@Sheeproid
Sheeproid enabled auto-merge (squash) February 10, 2026 11:32
@github-actions

github-actions Bot commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ Docker image ready for 12beaa5 (built in 38s)

⚠️ Warning: does not support ARM (ARM images are built on release only - not on every PR)

Use this tag to pull the image for testing.

📋 Copy commands

⚠️ Temporary images are deleted after 30 days. Copy to a permanent registry before using them:

gcloud auth configure-docker us-central1-docker.pkg.dev
docker pull us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:12beaa5
docker tag us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:12beaa5 me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:12beaa5
docker push me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:12beaa5

Patch Helm values in one line (choose the chart you use):

HolmesGPT chart:

helm upgrade --install holmesgpt ./helm/holmes \
  --set registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set image=holmes-dev:12beaa5

Robusta wrapper chart:

helm upgrade --install robusta robusta/robusta \
  --reuse-values \
  --set holmes.registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set holmes.image=holmes-dev:12beaa5

@coderabbitai

coderabbitai Bot commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

Walkthrough

Adds two optional ServiceNow config fields in docs and a configurable health_check_table in the ServiceNow Tables toolset; updates the health-check call path to use the configured table and improves HTTP/connection error messages. Tests adjust transformer registration/cleanup handling.

Changes

Cohort / File(s) Summary
Documentation
docs/data-sources/builtin-toolsets/servicenow.md
Add api_key_header and health_check_table to examples and new "Optional Fields" table (defaults: x-sn-apikey, sys_user).
ServiceNow Health Check Implementation
holmes/plugins/toolsets/servicenow_tables/servicenow_tables.py
Add health_check_table: str to ServiceNowTablesConfig; change _perform_health_check to accept table_name and query the configured table; update callers to pass health_check_table; enhance HTTP 401/403 and connection error messages with status and response/exception details.
Integration Tests
tests/integration/test_kubernetes_transformer_execution.py
Preserve and restore any pre-existing llm_summarize transformer during setup/teardown by saving self.original_llm_summarize; unregister/register accordingly and ensure cleanup of MockSummarizeTransformer.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Config as Config (ServiceNowTablesConfig)
participant Plugin as ServiceNowTablesPlugin
participant SN as ServiceNow API
Config->>Plugin: provide health_check_table
Plugin->>SN: GET /api/now/table/{table_name}?sysparm_limit=1
SN-->>Plugin: 200 OK / data or 401/403 error
Plugin-->>Config: return health-check result (success/failure and message)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • ServiceNow all Tables tool #1097: Modifies the same ServiceNow Tables toolset and health-check logic (related changes to ServiceNowTablesConfig and _perform_health_check).

Suggested reviewers

  • moshemorad
  • RoiGlinik
🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding a health_check_table configuration field to ServiceNow configuration with corresponding documentation and implementation updates.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments
tests/integration/test_kubernetes_transformer_execution.py (1)

66-91: Consider extracting shared setup/teardown into a base class or pytest fixture.

Both TestKubernetesTransformerExecution and TestTransformerPerformanceMetrics have identical save/restore logic for the transformer registry. A shared mixin or conftest fixture would reduce duplication.

Also, both classes reach into registry._transformers (a private attribute). If the registry API doesn't expose a getter, it might be worth adding a public registry.get(name) method to avoid coupling tests to internal structure.

Also applies to: 515-540

Tip

We've launched Issue Planner and it is currently in beta. Please try it out and share your feedback on Discord!


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

🔬 CLI Performance Benchmark

🟡 Startup Time (no LLM)

Measures holmes version execution time (imports + initialization)

Metric PR Master Change
Cold Start 10.28s 10.49s -2.0%
Warm Mean 4.86s 4.95s -1.8%
Warm Min 4.85s 4.91s
Warm Max 4.89s 5.06s

🟡 Full CLI with LLM

Measures holmes ask execution time (OpenRouter + Haiku 4.5)

Metric PR Master Change
Cold Start 28.86s 34.12s -15.4%
Warm Mean 8.25s 7.89s +4.6%
Warm Min 8.00s 7.78s
Warm Max 8.70s 8.00s

PR: 12beaa51 | Master: bee28de9 | Iterations: 5

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@holmes/plugins/toolsets/servicenow_tables/servicenow_tables.py`:
- Line 91: The _perform_health_check method is missing a type annotation for its
table_name parameter; update the function signature of
_perform_health_check(self, table_name) to include the type hint (table_name:
str) so it reads _perform_health_check(self, table_name: str) -> Tuple[bool,
str], ensuring imports for Tuple remain valid and mypy/type-checking passes.
- Around line 118-122: The returned error message in the except block catching
requests.exceptions.ConnectionError contains an accidental double space before
"Full error" — update the f-string in the except clause (the return that returns
(False, f"...")) to remove the extra space so the sentence reads "...unknown).
Full error: {str(e)}" (adjust the f-string around self.config.api_url and str(e)
accordingly).
🧹 Nitpick comments (2)
holmes/plugins/toolsets/servicenow_tables/servicenow_tables.py (2)

93-97: Stale comment: still references sys_db_object but table is now dynamic.

The comment on line 94 says "Query sys_db_object table" but the table is now configurable via the table_name parameter.

Proposed fix
-            # Query sys_db_object table with minimal data
+            # Query the specified table with minimal data to verify connectivity

100-100: Consider using !s conversion flag instead of str() in f-strings.

Ruff (RUF010) flags f"...{str(e)}" — the idiomatic form is f"...{e!s}". This applies to lines 121 (and line 89, though outside this range). The success message on line 100 is also flagged by TRY300 suggesting it move to an else block, though that's stylistic.

Proposed fix (line 121)
-                f"Failed to connect to ServiceNow instance at {self.config.api_url if self.config else 'unknown'}.  Full error: {str(e)}",
+                f"Failed to connect to ServiceNow instance at {self.config.api_url if self.config else 'unknown'}.  Full error: {e!s}",

Also applies to: 106-106, 111-111, 121-121

Comment thread holmes/plugins/toolsets/servicenow_tables/servicenow_tables.py
Comment thread holmes/plugins/toolsets/servicenow_tables/servicenow_tables.py
arikalon1
arikalon1 previously approved these changes Feb 10, 2026

@arikalon1 arikalon1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work

…erformanceMetrics teardown

  TestTransformerPerformanceMetrics.teardown_method was not restoring the
  original llm_summarize transformer after replacing it with a mock. When
  pytest-xdist scheduled test_yaml_transformer_parsing on the same worker
  after this class, the registry was missing llm_summarize, causing the
  YAML parser to silently drop transformer configs.

Signed-off-by: Tomer Keshet <tomer@robusta.dev>
@Sheeproid
Sheeproid merged commit 7166325 into master Feb 10, 2026
18 of 20 checks passed
@Sheeproid
Sheeproid deleted the servicenow-check branch February 10, 2026 15:54
moshemorad pushed a commit that referenced this pull request Feb 22, 2026
Signed-off-by: Tomer Keshet <tomer@robusta.dev>
Signed-off-by: Mohse Morad <moshemorad12340@gmail.com>
moshemorad pushed a commit that referenced this pull request Feb 22, 2026
Signed-off-by: Tomer Keshet <tomer@robusta.dev>
Signed-off-by: Mohse Morad <moshemorad12340@gmail.com>
@coderabbitai coderabbitai Bot mentioned this pull request Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants