Skip to content

Questions around Alloy Config for Prometheus Integration #4283

@pkutaj

Description

@pkutaj

Hello team,

I have questions around the Alloy config snippet available at

https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/cloud/manage/monitoring/prometheus.md?plain=1#L264-L280

...it seems to me that putting

targets = [{
		__address__ = "https://api.clickhouse.cloud/v1/organizations/<org_id>/prometheus",
	}]

into the config gives me the following error

ts=2025-08-14T12:59:21.306789263Z level=error msg="Creating target failed" component_path=/ component_id=prometheus.scrape.clickhouse_cloud scrape_pool=prometheus.scrape.clickhouse_cloud err="instance 0 in group prometheus.scrape.clickhouse_cloud_part_17241709254077376921: \"https://api.clickhouse.cloud/v1/organizations/e957a5f7-43e3-4c05-ad5a-d02b2dcd0593/prometheus\" is not a valid hostname"

With the use of ClaudeAI, I've put together this config working for Grafana running on my localhost — note that the __address__ takes only the domain name, not the path. Not sure if this is a bug in the docs, or something else, but I thought I'd share the finding

// Scrape ClickHouse Cloud metrics
prometheus.scrape "clickhouse_cloud" {
  targets = [{
    __address__ = "api.clickhouse.cloud", 
  }]

  scheme = "https"
  metrics_path = "/v1/organizations/e957a5f7-43e3-4c05-ad5a-d02b2dcd0593/prometheus"
  
  params = {
    "filtered_metrics" = ["true"],
  }

  honor_labels = true
  scrape_interval = "30s"
  scrape_timeout = "25s"

  basic_auth {
    username = "<API_key_id>"
    password = "<API_key_secret>"
  }

  forward_to = [prometheus.remote_write.clickhouse_metrics.receiver]
}

// Forward metrics to local Prometheus
prometheus.remote_write "clickhouse_metrics" {
  endpoint {
    url = "http://prometheus:9090/api/v1/write" 
    
    queue_config {
      capacity = 10000
      max_shards = 1000
      batch_send_deadline = "5s"
    }
  }
}

A final point is that perhaps the Grafana endpoint in the docs look hardcoded in the docs and should be commented on as needing to be edited.
But that's perhaps self-explanatory as well - and as said, it's just observations what works for me.
Thanks!

Image

https://clickhouse.com/docs/integrations/prometheus#integrating-with-grafana

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1integrationsNeeds attention from the integrations dev team

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions