Add firewall detection and user-friendly error handling for Supabase sign-in

CLAUDE.md

@@ -855,7 +855,7 @@ When writing documentation in the `docs/` directory:
 
 ## Robusta Platform/API URLs in Docs
 
-Whenever you reference `api.robusta.dev` or `platform.robusta.dev` in a docs page, use the `robusta-region` custom fence so readers can pick US/EU/AP. **Never hardcode a single region** — the Robusta platform is hosted in multiple regions and a bare `api.robusta.dev` link silently breaks for EU/AP users.
+Whenever you reference `api.robusta.dev`, `platform.robusta.dev`, or `sp.robusta.dev` (the Supabase host) in a docs page, use the `robusta-region` custom fence so readers can pick US/EU/AP. **Never hardcode a single region** — the Robusta platform is hosted in multiple regions and a bare `api.robusta.dev` link silently breaks for EU/AP users.
 
 The fence (defined in `docs/custom_fences.py`, registered in `mkdocs.yml`) takes a US URL as input and emits a three-tab picker with the domain rewritten per region. Pick the input shape that matches your context:
 
@@ -886,6 +886,6 @@ The fence (defined in `docs/custom_fences.py`, registered in `mkdocs.yml`) takes
     ```
     ````
 
-Author the URL once using the US domain (`api.robusta.dev` / `platform.robusta.dev`); the fence handles the `.eu` / `.ap` rewrites. If you add a new region or rename one, update `ROBUSTA_REGIONS` in `docs/custom_fences.py` — that single change propagates to every page.
+Author the URL once using the US domain (`api.robusta.dev` / `platform.robusta.dev` / `sp.robusta.dev`); the fence handles the `.eu` / `.ap` rewrites. The set of rewritten hosts lives in `ROBUSTA_DOMAIN_RE` in `docs/custom_fences.py` — add a host there if you need another subdomain covered. If you add a new region or rename one, update `ROBUSTA_REGIONS` in the same file — that single change propagates to every page.
 
-Existing usages (greppable starting points): `docs/ai-providers/robusta-ai.md`, `docs/installation/ui-installation.md`, `docs/reference/environment-variables.md`, `docs/data-sources/builtin-toolsets/coralogix-logs.md`, `docs/data-sources/builtin-toolsets/kubernetes-mcp.md`.
+Existing usages (greppable starting points): `docs/ai-providers/robusta-ai.md`, `docs/installation/ui-installation.md`, `docs/reference/environment-variables.md`, `docs/reference/troubleshooting.md`, `docs/data-sources/builtin-toolsets/coralogix-logs.md`, `docs/data-sources/builtin-toolsets/kubernetes-mcp.md`.

docs/custom_fences.py

@@ -4,8 +4,8 @@
 Fences available:
 - yaml-toolset-config: Creates 3 tabs (Holmes CLI, Holmes Helm Chart, Robusta Helm Chart) for toolset configurations
 - yaml-helm-values: Creates 2 tabs (Holmes Helm Chart, Robusta Helm Chart) for Helm-only configurations like permissions
-- robusta-region: Creates 3 tabs (US, EU, AP) for any text containing api.robusta.dev or platform.robusta.dev. Plain
-  URLs render as code blocks; markdown links `[text](url)` render as clickable links.
+- robusta-region: Creates 3 tabs (US, EU, AP) for any text containing api.robusta.dev, platform.robusta.dev, or
+  sp.robusta.dev. Plain URLs render as code blocks; markdown links `[text](url)` render as clickable links.
 """
 
 import html
@@ -15,12 +15,12 @@
 import yaml  # type: ignore
 
 ROBUSTA_REGIONS = (("US", ""), ("EU", "eu"), ("AP", "ap"))
-ROBUSTA_DOMAIN_RE = re.compile(r"\b(api|platform)\.robusta\.dev\b")
+ROBUSTA_DOMAIN_RE = re.compile(r"\b(api|platform|sp)\.robusta\.dev\b")
 MARKDOWN_LINK_RE = re.compile(r"^\[([^\]]+)\]\(([^)\s]+)\)(\{[^}]*\})?$")
 
 
 def _rewrite_robusta_domain(text: str, region_infix: str) -> str:
-    """Rewrite api.robusta.dev / platform.robusta.dev to the regional variant."""
+    """Rewrite api/platform/sp .robusta.dev to the regional variant."""
     if not region_infix:
         return text
     return ROBUSTA_DOMAIN_RE.sub(rf"\1.{region_infix}.robusta.dev", text)
@@ -140,8 +140,8 @@ def helm_tabs_fence_format(source, language, css_class, options, md, **kwargs):
 
 def robusta_region_fence_format(source, language, css_class, options, md, **kwargs):
     """
-    Render the source as three tabs (US, EU, AP), rewriting `api.robusta.dev`
-    and `platform.robusta.dev` to the regional subdomain in each tab.
+    Render the source as three tabs (US, EU, AP), rewriting `api.robusta.dev`,
+    `platform.robusta.dev` and `sp.robusta.dev` to the regional subdomain in each tab.
 
     Auto-detects two input shapes:
 

docs/reference/troubleshooting.md

@@ -73,6 +73,29 @@ export LLM_EXTRA_STRIP_MESSAGE_FIELDS="provider_specific_fields"
 
 Replace the value with whichever field is named in your error message. Multiple fields can be passed, e.g. `"provider_specific_fields,reasoning_content"`.
 
+## 7. Startup Fails with `Connection reset by peer` { #firewall-blocking-robusta-platform }
+
+HolmesGPT crashes on startup while signing in to the Robusta platform, with a traceback ending in:
+
+```text
+httpx.ConnectError: [Errno 104] Connection reset by peer
+```
+
+This means an **outbound firewall or egress policy is blocking traffic from your cluster to the Robusta platform**. The hostname resolves and the TLS certificate is valid, so it is not a DNS or certificate problem — the connection itself is being reset or refused.
+
+**Solution:**
+
+Allow outbound HTTPS (port 443) from the HolmesGPT pod to the Robusta platform — i.e. allowlist the `robusta.dev` domain (`*.robusta.dev`), which covers the `api.*` and `sp.*` subdomains across all regions.
+
+To confirm the block, run the one-off pod below. It **auto-detects the Holmes pod's namespace and image** and curls the platform from a fresh pod — the HolmesGPT pod itself crashes on this error (`CrashLoopBackOff`), so `kubectl exec` into it won't work. Reusing Holmes's own image means nothing new is pulled (the same firewall may also block image pulls) and it shares Holmes's CA and network config. Just pick your region — a firewall block shows `Connection reset by peer`, while a reachable endpoint returns JSON:
+
+```robusta-region {lang=bash}
+read -r NS IMG <<<"$(kubectl get pods -A -l app=holmes -o jsonpath='{.items[0].metadata.namespace} {.items[0].spec.containers[0].image}')"
+kubectl run holmes-egress-check --rm -it --restart=Never -n "$NS" --image="$IMG" --command -- curl -vk https://sp.robusta.dev/auth/v1/health
+```
+
+If the same logs also show a LiteLLM warning about failing to fetch the model cost map from `raw.githubusercontent.com`, that is the same firewall blocking GitHub egress — point Holmes at a region-local mirror with [`LITELLM_MODEL_COST_MAP_URL`](environment-variables.md#litellm_model_cost_map_url).
+
 ---
 
 ## Still stuck?

holmes/core/supabase_dal.py

@@ -127,6 +127,14 @@ class RobustaToken(BaseModel):
     password: str
 
 
+# Troubleshooting guide for an outbound firewall blocking egress to the Robusta
+# platform (surfaces as a connection reset during sign-in). Linked from the log
+# and exception so users can find the fix.
+FIREWALL_TROUBLESHOOTING_URL = (
+    "https://holmesgpt.dev/reference/troubleshooting/#firewall-blocking-robusta-platform"
+)
+
+
 class SupabaseDnsException(Exception):
     def __init__(self, error: Exception, url: str):
         message = (
@@ -138,6 +146,23 @@ def __init__(self, error: Exception, url: str):
         super().__init__(message)
 
 
+class SupabaseConnectionException(Exception):
+    """Raised when Holmes cannot open a connection to the Robusta platform.
+
+    Almost always an outbound firewall / egress policy blocking traffic to the
+    Robusta platform (not a DNS or TLS certificate problem). The actionable
+    guidance - allowlist '*.robusta.dev' plus the docs link - is logged at
+    WARNING right before this is raised, so the exception message itself stays a
+    thin technical wrapper around the underlying connection error.
+    """
+
+    def __init__(self, error: Exception, url: str):
+        super().__init__(
+            f"Could not connect to the Robusta platform at {url} "
+            f"({error.__class__.__name__}: {error})"
+        )
+
+
 class SupabaseDal:
     def __init__(self, cluster: str):
         self.enabled = self.__init_config()
@@ -291,6 +316,33 @@ def sign_in(self) -> str:
                 ]
             ):
                 raise SupabaseDnsException(e, self.url) from e
+            if isinstance(e, (ConnectionError, TimeoutError)) or any(
+                conn_indicator in error_msg
+                for conn_indicator in [
+                    "connection reset by peer",
+                    "connection reset",
+                    "connection refused",
+                    "connection aborted",
+                    "connection timed out",
+                    "network is unreachable",
+                    "no route to host",
+                    "errno 104",  # ECONNRESET
+                    "errno 111",  # ECONNREFUSED
+                ]
+            ):
+                # The platform resolved but refused/reset the connection - almost
+                # always an outbound firewall. Log the full actionable guidance at
+                # WARNING (not ERROR, so it doesn't raise a Sentry alert) before
+                # raising; the exception below stays a thin technical wrapper.
+                logging.warning(
+                    "Could not connect to the Robusta platform at %s. This is "
+                    "usually an outbound firewall blocking egress to the platform - "
+                    "allowlist outbound HTTPS to '*.robusta.dev'. See %s for "
+                    "troubleshooting steps.",
+                    self.url,
+                    FIREWALL_TROUBLESHOOTING_URL,
+                )
+                raise SupabaseConnectionException(e, self.url) from e
             raise
 
     def get_resource_recommendation(

tests/core/test_supabase_dal.py

@@ -1,17 +1,111 @@
 """Unit tests for SupabaseDal.get_resource_recommendation method."""
 
+import logging
 from unittest.mock import Mock, patch
 
 import pytest
 from postgrest.exceptions import APIError as PGAPIError
 
 from holmes.core.supabase_dal import (
+    FIREWALL_TROUBLESHOOTING_URL,
     GROUPED_ISSUES_TABLE,
     ISSUES_TABLE,
+    SupabaseConnectionException,
     SupabaseDal,
+    SupabaseDnsException,
 )
 
 
+class TestSignIn:
+    """Tests for SupabaseDal.sign_in() error classification.
+
+    A firewall / egress policy that blocks the cluster from reaching the Robusta
+    platform surfaces as a connection reset/refused during sign-in. Holmes should
+    convert that into a SupabaseConnectionException whose message points the user
+    at their firewall, instead of leaking a raw httpx traceback. Genuine auth
+    errors must still propagate unchanged.
+    """
+
+    @pytest.fixture
+    def mock_dal(self):
+        with patch("holmes.core.supabase_dal.create_client"):
+            dal = SupabaseDal(cluster="test-cluster")
+            dal.enabled = True
+            dal.client = Mock()
+            dal.url = "https://sp.eu.robusta.dev"
+            dal.email = "user@example.com"
+            dal.password = "secret"
+            return dal
+
+    def test_connection_reset_raises_firewall_exception(self, mock_dal, caplog):
+        # The exact error Aviva hit at startup (ROB-273): httpx surfaces the
+        # firewall block as "[Errno 104] Connection reset by peer".
+        mock_dal.client.auth.sign_in_with_password.side_effect = Exception(
+            "[Errno 104] Connection reset by peer"
+        )
+
+        with caplog.at_level(logging.WARNING):
+            with pytest.raises(SupabaseConnectionException) as exc_info:
+                mock_dal.sign_in()
+
+        # The exception stays a thin technical wrapper - it names the platform and
+        # the underlying error but carries none of the actionable guidance.
+        message = str(exc_info.value)
+        assert "Robusta platform" in message
+        assert "curl" not in message
+        assert "*.robusta.dev" not in message
+        assert FIREWALL_TROUBLESHOOTING_URL not in message
+
+        # All the firewall guidance - cause, the allowlist fix, and the docs link -
+        # is logged at WARNING (not ERROR, so it doesn't raise a Sentry alert)
+        # before the exception is raised.
+        warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
+        assert any("firewall" in r.getMessage().lower() for r in warnings)
+        assert any("*.robusta.dev" in r.getMessage() for r in warnings)
+        assert any(FIREWALL_TROUBLESHOOTING_URL in r.getMessage() for r in warnings)
+
+    def test_connection_refused_raises_firewall_exception(self, mock_dal):
+        mock_dal.client.auth.sign_in_with_password.side_effect = (
+            ConnectionRefusedError("[Errno 111] Connection refused")
+        )
+        with pytest.raises(SupabaseConnectionException):
+            mock_dal.sign_in()
+
+    def test_timeout_raises_firewall_exception(self, mock_dal):
+        mock_dal.client.auth.sign_in_with_password.side_effect = TimeoutError(
+            "connection timed out"
+        )
+        with pytest.raises(SupabaseConnectionException):
+            mock_dal.sign_in()
+
+    def test_dns_error_still_raises_dns_exception(self, mock_dal):
+        mock_dal.client.auth.sign_in_with_password.side_effect = Exception(
+            "Temporary failure in name resolution"
+        )
+        with pytest.raises(SupabaseDnsException):
+            mock_dal.sign_in()
+
+    def test_auth_error_is_not_wrapped(self, mock_dal):
+        # A genuine credential error is not a connectivity/firewall problem;
+        # wrapping it would mislead the user, so it must propagate unchanged.
+        original = ValueError("Invalid login credentials")
+        mock_dal.client.auth.sign_in_with_password.side_effect = original
+        with pytest.raises(ValueError) as exc_info:
+            mock_dal.sign_in()
+        assert exc_info.value is original
+
+    def test_successful_sign_in_returns_user_id(self, mock_dal):
+        session = Mock(access_token="access-token", refresh_token="refresh-token")
+        res = Mock(session=session, user=Mock(id="user-123"))
+        mock_dal.client.auth.sign_in_with_password.return_value = res
+
+        assert mock_dal.sign_in() == "user-123"
+        mock_dal.client.auth.set_session.assert_called_once_with(
+            "access-token", "refresh-token"
+        )
+        mock_dal.client.postgrest.auth.assert_called_once_with("access-token")
+
+
 class TestIsRealtimeEnabled:
     """Tests for SupabaseDal.is_realtime_enabled()."""