ROB-3020 shorten mock file names to avoid errors. revert git command change

[RoiGlinik]

Summary by CodeRabbit

• Tests

  • Removed multiple outdated test fixtures for New Relic NRQL query operations, including queries for crashpod logs, Prometheus logs, and related infrastructure data
  • Added test fixture for TCP connectivity checks against Prometheus server

• Chores

  • Updated CI/CD workflow configuration to streamline Windows build process

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[netlify]

✅ Deploy Preview for holmes-docs ready!

Name	Link
🔨 Latest commit	35af208
🔍 Latest deploy log	https://app.netlify.com/projects/holmes-docs/deploys/696cf17c7065460008b7c3b7
😎 Deploy Preview	https://deploy-preview-1375--holmes-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

✅ Results of HolmesGPT evals

Automatically triggered by commit 35af208 on branch ROB-3020-shorten-mock-files-names

View workflow logs

Results of HolmesGPT evals

• ask_holmes: 9/9 test cases were successful, 0 regressions

Status	Test case	Time	Turns	Tools	Cost
✅	09_crashpod	27.9s ↓14%	5	12	$0.1563
✅	101_loki_historical_logs_pod_deleted	43.8s ↓21%	6	14	$0.1973
✅	111_pod_names_contain_service	37.5s ±0%	7	16	$0.1852
✅	12_job_crashing	51.6s ↑16%	10	23	$0.2535
✅	162_get_runbooks	40.9s ±0%	7	13	$0.1998
✅	176_network_policy_blocking_traffic_no_runbooks	43.0s ↑21%	8	16	$0.1926
✅	24_misconfigured_pvc	33.7s ±0%	7	16	$0.1710
✅	43_current_datetime_from_prompt	3.2s ±0%	1	—	$0.0618
✅	61_exact_match_counting	9.8s ±0%	3	3	$0.0859
	Total	32.4s avg	6.0 avg	14.1 avg	$1.5035

Time/Cost columns show % change vs historical average (↑slower/costlier, ↓faster/cheaper). Changes under 10% shown as ±0%.

Historical Comparison Details

Filter: excluding branch 'ROB-3020-shorten-mock-files-names'

Status: Success - 21 test/model combinations loaded

Experiments compared (30):

• root-master-k=124b_new_relic_multi_account_alert_prompt-20260118_163139 (branch: master)
• github-21112880869.2088.1 (branch: master)
• github-21112820511.2087.1 (branch: ROB-3020-windows-build-failed)
• ...and 27 more

Comparison indicators:

• ±0% — diff under 10% (within noise threshold)
• ↑N%/↓N% — diff 10-25%
• ↑N%/↓N% — diff over 25% (significant)

📖 Legend

Icon	Meaning
✅	The test was successful
➖	The test was skipped
⚠️	The test failed but is known to be flaky or known to fail
🚧	The test had a setup failure (not a code regression)
🔧	The test failed due to mock data issues (not a code regression)
🚫	The test was throttled by API rate limits/overload
❌	The test failed and should be fixed before merging the PR

🔄 Re-run evals manually

⚠️ Warning: /eval comments always run using the workflow from master, not from this PR branch. If you modified the GitHub Action (e.g., added secrets or env vars), those changes won't take effect.

To test workflow changes, use the GitHub CLI or Actions UI instead:

gh workflow run eval-regression.yaml --repo HolmesGPT/holmesgpt --ref ROB-3020-shorten-mock-files-names -f markers=regression -f filter=

---

Option 1: Comment on this PR with /eval:

/eval
markers: regression

Or with more options (one per line):

/eval
model: gpt-4o
markers: regression
filter: 09_crashpod
iterations: 5

Run evals on a different branch (e.g., master) for comparison:

/eval
branch: master
markers: regression

Option	Description
model	Model(s) to test (default: same as automatic runs)
markers	Pytest markers (no default - runs all tests!)
filter	Pytest -k filter (use /list to see valid eval names)
iterations	Number of runs, max 10
branch	Run evals on a different branch (for cross-branch comparison)

Quick re-run: Use /rerun to re-run the most recent /eval on this PR with the same parameters.

Option 2: Trigger via GitHub Actions UI → "Run workflow"

🏷️ Valid markers

benchmark, chain-of-causation, compaction, context_window, coralogix, counting, database, datadog, datetime, easy, elasticsearch, embeds, frontend, grafana-dashboard, hard, kafka, kubernetes, leaked-information, logs, loki, medium, metrics, network, newrelic, no-cicd, numerical, one-test, port-forward, prometheus, question-answer, regression, runbooks, slackbot, storage, toolset-limitation, traces, transparency

---

Commands: /eval · /rerun · /list

CLI: gh workflow run eval-regression.yaml --repo HolmesGPT/holmesgpt --ref ROB-3020-shorten-mock-files-names -f markers=regression -f filter=

[coderabbitai]

Walkthrough

The pull request removes a Windows-specific Git configuration step from the build workflow and deletes multiple obsolete New Relic NRQL query test fixtures while adding a new TCP connectivity check test fixture.

Changes

Cohort / File(s)	Summary
Workflow Configuration .github/workflows/build-binaries-and-brew.yaml	Removed Windows "git config --system core.longpaths true" step from Install dependencies task
New Relic Test Fixtures (Removed) tests/llm/fixtures/test_ask_holmes/124a_new_relic_multi_account_account_name/newrelic_execute_nrql_query7481278_Find_crashpod_logs_*, newrelic_execute_nrql_query7481278_Get_crashpod_container_logs_*	Deleted NRQL query execution fixtures for crashpod log queries with metadata and response payloads
Prometheus Pod Log Fixtures (Removed) tests/llm/fixtures/test_ask_holmes/124b_new_relic_multi_account_alert_prompt/newrelic_execute_nrql_query6358011_Prometheus_pod_logs_*, newrelic_execute_nrql_query6358011_Prometheus_server_pod_logs_*, newrelic_execute_nrql_query6358011_Prometheus_server_pod_status_*	Deleted NRQL query fixtures for Prometheus server logs and status queries with empty result sets
Non-EOF Messages Fixture (Removed) tests/llm/fixtures/test_ask_holmes/124c_new_relic_multi_account_default/newrelic_execute_nrql_query6358011_Get_non-EOF_messages_*	Removed NRQL query fixture for filtering non-EOF log messages with metadata and response envelope
TCP Connectivity Test Fixture (Added) tests/llm/fixtures/test_ask_holmes/124b_new_relic_multi_account_alert_prompt/tcp_checkprometheus-server.default.svc.cluster.local_9090.txt	Added TCP connectivity check fixture for Prometheus server connectivity verification with failure response

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• ROB-3020 add long file name support in windows #1368: Directly opposes this change by adding the Windows "git config --system core.longpaths true" step that this PR removes from the same workflow file.
• Fix Release Build #706: Modifies the same .github/workflows/build-binaries-and-brew.yaml file with OS-specific build job steps.

Suggested reviewers

• Sheeproid
• nherment

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the two main changes: shortening mock file names and reverting a git command change, which aligns with the changeset modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

✅ Docker image ready for 28a1e79 (built in 4m 5s)

• us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:28a1e79

⚠️ Warning: does not support ARM (ARM images are built on release only - not on every PR)

Use this tag to pull the image for testing.

📋 Copy commands

⚠️ Temporary images are deleted after 30 days. Copy to a permanent registry before using them:

gcloud auth configure-docker us-central1-docker.pkg.dev
docker pull us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:28a1e79
docker tag us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:28a1e79 me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:28a1e79
docker push me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:28a1e79

Patch Helm values in one line (choose the chart you use):

HolmesGPT chart:

helm upgrade --install holmesgpt ./helm/holmes \
  --set registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set image=holmes-dev:28a1e79

Robusta wrapper chart:

helm upgrade --install robusta robusta/robusta \
  --reuse-values \
  --set holmes.registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set holmes.image=holmes-dev:28a1e79

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In
`@tests/llm/fixtures/test_ask_holmes/124b_new_relic_multi_account_alert_prompt/tcp_checkprometheus-server.default.svc.cluster.local_9090.txt`:
- Around line 1-3: The filename is missing an underscore between the tool name
and host; rename the file from
tcp_checkprometheus-server.default.svc.cluster.local_9090.txt to
tcp_check_prometheus-server.default.svc.cluster.local_9090.txt so it follows the
directory naming convention (matching other fixtures like
newrelic_execute_nrql_query6358011_Prometheus_pod_logs.txt); update any
references to this filename in tests or fixtures (search for
tcp_checkprometheus-server.default.svc.cluster.local_9090) to use the new name.