prom2json

A tool to scrape a Prometheus client and dump the result as JSON.

Background

(Pre-)historically, Prometheus clients were able to expose metrics as JSON. For various reasons, the JSON exposition format was deprecated.

Usually, scraping of a Prometheus client is done by the Prometheus server, which preferably happens with the protocol buffer format. Sometimes, a human being needs to inspect what a Prometheus clients exposes. In that case, the text format is used (which is otherwise meant to allow simplistic clients like shell scripts to expose metrics to a Prometheus server).

However, some users wish to scrape Prometheus clients with programs other than the Prometheus server. Those programs would usually use the protocol buffer format, but for small ad hoc programs, that is too much of an (programming) overhead. JSON comes in handy for these use-cases, as many languages offer tooling for JSON parsing.

To avoid maintaining a JSON format in all client libraries, the prom2json tool has been created, which scrapes a Prometheus client in protocol buffer or text format and dumps the result as JSON to stdout.

Usage

Installing and building:

$ GO111MODULE=on go install github.com/prometheus/prom2json/cmd/prom2json@latest

Running:

$ prom2json http://my-prometheus-client.example.org:8080/metrics
$ curl http://my-prometheus-client.example.org:8080/metrics | prom2json
$ prom2json /tmp/metrics.prom

Running with TLS client authentication:

$ prom2json --cert=/path/to/certificate --key=/path/to/key http://my-prometheus-client.example.org:8080/metrics

Running without TLS validation (insecure, do not use in production!):

$ prom2json --accept-invalid-cert https://my-prometheus-client.example.org:8080/metrics

Advanced HTTP through curl:

$ curl -XPOST -H 'X-CSRFToken: 1234567890abcdef' --connect-timeout 60 'https://username:[email protected]:8080/metrics' | prom2json

This will dump the JSON to stdout. Note that the dumped JSON is not using the deprecated JSON format as specified in the Prometheus exposition format reference. The created JSON uses a format much closer in structure to the protocol buffer format. It is only used by the prom2json tool and has no significance elsewhere. See below for a description.

A typical use-case is to pipe the JSON format into a tool like jq to run a query over it. That looked like the following when the clients still supported the deprecated JSON format:

$ curl http://my-prometheus-client.example.org:8080/metrics | jq .

Now simply use prom2json instead of curl (and change the query syntax according to the changed JSON format generated by prom2json):

$ prom2json http://my-prometheus-client.example.org:8080/metrics | jq .

Example query to retrieve the number of metrics in the http_requests_total metric family (only works with the new format):

$ prom2json http://my-prometheus-client.example.org:8080/metrics | jq '.[]|select(.name=="http_requests_total")|.metrics|length'

Example input from stdin:

$ curl http://my-prometheus-client.example.org:8080/metrics | grep http_requests_total | prom2json

JSON format

Note that all numbers are encoded as strings. Some parsers want it that way. Also, Prometheus allows sample values like NaN or +Inf, which cannot be encoded as JSON numbers.

A histogram is formatted as a native histogram if it has at least one span. It is then formatted in a similar way as the Prometehus query API does it.

[
  {
    "name": "http_request_duration_microseconds",
    "help": "The HTTP request latencies in microseconds.",
    "type": "SUMMARY",
    "metrics": [
      {
        "labels": {
          "method": "get",
          "handler": "prometheus",
          "code": "200"
        },
        "quantiles": {
          "0.99": "67542.292",
          "0.9": "23902.678",
          "0.5": "6865.718"
        },
        "count": "743",
        "sum": "6936936.447000001"
      },
      {
        "labels": {
          "method": "get",
          "handler": "prometheus",
          "code": "400"
        },
        "quantiles": {
          "0.99": "3542.9",
          "0.9": "1202.3",
          "0.5": "1002.8"
        },
        "count": "4",
        "sum": "345.01"
      }
    ]
  },
  {
    "name": "roshi_select_call_count",
    "help": "How many select calls have been made.",
    "type": "COUNTER",
    "metrics": [
      {
        "value": "1063110"
      }
    ]
  },
  {
    "name": "http_request_duration_seconds",
    "type": "HISTOGRAM",
    "help": "This is a native histogram.",
    "metrics": [
      {
        "labels": {
        "method": "GET",
        },
        "buckets": [
          [
            0,
            "17.448123722644123",
            "19.027313840043536",
            "139"
          ],
          [
            0,
            "19.027313840043536",
            "20.749432874416154",
            "85"
          ],
          [
            0,
            "20.749432874416154",
            "22.62741699796952",
            "70"
          ],
        ],
        "count": "1000",
        "sum": "29969.50000000001"
      }
    ]
  },
  {
    "name": "some_weird_normal_distribution",
    "type": "HISTOGRAM",
    "help": "This is a classic histogram.",
    "metrics": [
      {
        "buckets": {
          "-0.0001899999999999998": "17",
          "-0.0002899999999999998": "6",
          "-0.0003899999999999998": "2",
          "-0.0004899999999999998": "2",
          "-0.0005899999999999998": "0",
          "-0.0006899999999999999": "0",
          "-0.0007899999999999999": "0",
          "-0.00089": "0",
          "-0.00099": "0",
          "-8.999999999999979e-05": "33",
          "0.00011000000000000022": "75",
          "0.00021000000000000023": "92",
          "0.0003100000000000002": "100",
          "0.0004100000000000002": "103",
          "0.0005100000000000003": "105",
          "0.0006100000000000003": "106",
          "0.0007100000000000003": "107",
          "0.0008100000000000004": "107",
          "0.0009100000000000004": "107",
          "1.0000000000000216e-05": "50"
        },
        "count": "107",
        "sum": "0.001792103516591124"
      }
    ]
  }
]

Using Docker

You can deploy this tool using the prom/prom2json Docker image.

For example:

docker pull prom/prom2json

docker run --rm -ti prom/prom2json http://my-prometheus-client.example.org:8080/metrics