Unset body_size_limit option causes panic

[vorotech]

Host operating system

Linux ip-xxx-xxx-xxx-xxx 5.11.0-1019-aws #20~20.04.1-Ubuntu SMP Tue Sep 21 10:40:39 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

blackbox_exporter version: output of blackbox_exporter --version

master with the merged PR #836

What is the blackbox.yml module config.

modules:
  http_2xx:
    prober: http
    http:
      preferred_ip_protocol: ip4
  http_post_2xx:
    prober: http
    http:
      method: POST

Docker-compose service config.

version: "3"
services:
  blackbox_exporter:
    image: prom/blackbox-exporter:master
    container_name: blackbox_exporter
    volumes:
      - ./blackbox_exporter_config.yml:/config/blackbox.yml
    command: --config.file=/config/blackbox.yml
    ports:
      - 9115:9115
    restart: always

What did you do that produced an error?

Container is crashed on docker start with the panic.

If body_size_limit is set, than service starts without a crash.

What did you expect to see?

level=info ts=2021-10-19T19:48:05.267Z caller=main.go:224 msg="Starting blackbox_exporter" version="(version=0.19.0, branch=master, revision=e86980961f8e463288e31cac03255701cb1a11d1)"
level=info ts=2021-10-19T19:48:05.267Z caller=main.go:225 build_context="(go=go1.16.9, user=root@34250ccfd885, date=20211019-16:51:42)"
level=info ts=2021-10-19T19:48:05.267Z caller=main.go:237 msg="Loaded config file"
level=info ts=2021-10-19T19:48:05.268Z caller=main.go:385 msg="Listening on address" address=:9115
level=info ts=2021-10-19T19:48:05.268Z caller=tls_config.go:191 msg="TLS is disabled." http2=false

What did you see instead?

2021/10/19 19:29:27 http: panic serving 127.0.0.1:44030: runtime error: slice bounds out of range [:-9223372036854775808]
goroutine 11233 [running]:
net/http.(*conn).serve.func1(0xc00044b0e0)
  /usr/local/go/src/net/http/server.go:1804 +0x153
panic(0xb616a0, 0xc000036ff0)
  /usr/local/go/src/runtime/panic.go:971 +0x499
net/http.(*maxBytesReader).Read(0xc0003f4bc0, 0xc0002c6000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
  /usr/local/go/src/net/http/request.go:1147 +0x206
github.com/prometheus/blackbox_exporter/prober.(*byteCounter).Read(0xc00024b9b0, 0xc0002c6000, 0x2000, 0x2000, 0xc8a7a0, 0xc000528c30, 0x40a83f)
  /app/prober/http.go:230 +0x51
io.discard.ReadFrom(0xc8a960, 0xc00024b9b0, 0xb1cf80, 0xb2a401, 0x7fb35ee89838)
  /usr/local/go/src/io/io.go:598 +0x95
io.copyBuffer(0xc8bb40, 0x10c8680, 0xc8a960, 0xc00024b9b0, 0x0, 0x0, 0x0, 0x0, 0xc00036a800, 0x0)
  /usr/local/go/src/io/io.go:409 +0x357
io.Copy(...)
  /usr/local/go/src/io/io.go:382
github.com/prometheus/blackbox_exporter/prober.ProbeHTTP(0xc95c20, 0xc0004150e0, 0xc000435c80, 0x31, 0xc0001601c8, 0x4, 0x0, 0x0, 0x0, 0x0, ...)
  /app/prober/http.go:522 +0x44eb
main.probeHandler(0xc94b60, 0xc0001dc7e0, 0xc00036a000, 0xc00000e360, 0xc8a7a0, 0xc000145dd0, 0xc0000753b0)
  /app/main.go:130 +0x9ce
main.run.func5(0xc94b60, 0xc0001dc7e0, 0xc00036a000)
  /app/main.go:320 +0xb1
net/http.HandlerFunc.ServeHTTP(0xc00018a120, 0xc94b60, 0xc0001dc7e0, 0xc00036a000)
  /usr/local/go/src/net/http/server.go:2049 +0x44
net/http.(*ServeMux).ServeHTTP(0x1099d60, 0xc94b60, 0xc0001dc7e0, 0xc00036a000)
  /usr/local/go/src/net/http/server.go:2428 +0x1ad
net/http.serverHandler.ServeHTTP(0xc0001dc000, 0xc94b60, 0xc0001dc7e0, 0xc00036a000)
  /usr/local/go/src/net/http/server.go:2867 +0xa3
net/http.(*conn).serve(0xc00044b0e0, 0xc95c58, 0xc0003f4200)
  /usr/local/go/src/net/http/server.go:1932 +0x8cd
created by net/http.(*Server).Serve
  /usr/local/go/src/net/http/server.go:2993 +0x39b

[vorotech]

hi @mem
I noticed an issue introduced by last merged PR with introduced body_size_limit option

[mem]

Oops, sorry, looking

[mem]

Sorry about that, turns out the limiter in the http package tries to add 1 to the limit we specify and it ends up indexing an slice with a negative number.

I'm still tracking down why the existing tests didn't catch this, but I verified by hand that the linked fixes handles all the cases (negative, 0, positive but causing the request to hit the limit, large positive).