Crash in cgo DNS code #304

roman-vynar · 2016-04-12T11:56:46Z

Alertmanager crashes when notify by email fails:

[root@db1 ~]# ./alertmanager-0.1.1.linux-amd64/alertmanager -web.listen-address=:9093 -config.file=./alertmanager.yml -storage.path=./alertmanager-0.1.1.linux-amd64/data
alertmanager, version 0.1.1 (branch: release-0.1, revision: 0e541bf)
  build user:       root@8c44a0677215
  build date:       20160323-10:10:18
  go version:       go1.5.3
INFO[0000] Loading configuration file                    file=./alertmanager.yml source=main.go:125
WARN[0000] Notify attempt 1 failed: sending rcpt to: 550 5.1.1 <[email protected]>: Recipient address rejected: User unknown in local recipient table  source=notify.go:193
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7fd1825ed67d]

runtime stack:
runtime.throw(0xb8ef60, 0x2a)
    /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a
...

I expect it to log an error but not crash.

The text was updated successfully, but these errors were encountered:

juliusv · 2016-04-12T15:27:20Z

Do you have more of the stack trace, including at least the first goroutine? Does this happen every time?

roman-vynar · 2016-04-12T16:10:58Z

Looks like this WARN about bad email has nothing to do with the crash.
The crash is happening all the time randomly after sometime Alertmanager runs.

First:

[root@db1 ~]# ./alertmanager-0.1.1.linux-amd64/alertmanager -web.listen-address=:9093 -config.file=./alertmanager.yml -storage.path=./alertmanager-0.1.1.linux-amd64/data
alertmanager, version 0.1.1 (branch: release-0.1, revision: 0e541bf)
  build user:       root@8c44a0677215
  build date:       20160323-10:10:18
  go version:       go1.5.3
INFO[0000] Loading configuration file                    file=./alertmanager.yml source=main.go:125
WARN[0000] Notify attempt 1 failed: sending rcpt to: 550 5.1.1 <[email protected]>: Recipient address rejected: User unknown in local recipient table  source=notify.go:193
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7fd1825ed67d]

runtime stack:
runtime.throw(0xb8ef60, 0x2a)
    /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 81 [syscall, locked to thread]:
runtime.cgocall(0x806280, 0xc8200469d8, 0xc800000000)
    /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc820046980 sp=0xc820046950
net._C2func_getaddrinfo(0x7fd1740008c0, 0x0, 0xc820459830, 0xc820024c60, 0x0, 0x0, 0x0)
    ??:0 +0x4d fp=0xc8200469d8 sp=0xc820046980
net.cgoLookupIPCNAME(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/cgo_unix.go:111 +0x1ac fp=0xc820046b00 sp=0xc8200469d8
net.cgoLookupIP(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x6)
    /usr/local/go/src/net/cgo_unix.go:163 +0x56 fp=0xc820046b58 sp=0xc820046b00
net.lookupIP(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup_unix.go:67 +0x94 fp=0xc820046bc8 sp=0xc820046b58
net.glob.func15(0xbd4d40, 0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/hook.go:10 +0x4d fp=0xc820046c08 sp=0xc820046bc8
net.lookupIPMerge.func1(0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:68 +0x71 fp=0xc820046c88 sp=0xc820046c08
internal/singleflight.(*Group).doCall(0x100d770, 0xc8204540f0, 0xc8200c4c20, 0x9, 0xc820046e80)
    /usr/local/go/src/internal/singleflight/singleflight.go:93 +0x2c fp=0xc820046d38 sp=0xc820046c88
internal/singleflight.(*Group).Do(0x100d770, 0xc8200c4c20, 0x9, 0xc820046e80, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/internal/singleflight/singleflight.go:63 +0x284 fp=0xc820046da8 sp=0xc820046d38
net.lookupIPMerge(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:69 +0x9b fp=0xc820046eb8 sp=0xc820046da8
net.lookupIPDeadline(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:91 +0xde fp=0xc8200470f0 sp=0xc820046eb8
net.internetAddrList(0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /usr/local/go/src/net/ipsock.go:252 +0x6ee fp=0xc820047258 sp=0xc8200470f0
net.resolveAddrList(0xae4d60, 0x4, 0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0xc800000000, 0x0, 0x0, ...)
    /usr/local/go/src/net/dial.go:153 +0x466 fp=0xc820047330 sp=0xc820047258
net.(*Dialer).Dial(0xc820047558, 0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/dial.go:209 +0x124 fp=0xc820047510 sp=0xc820047330
net.Dial(0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/dial.go:186 +0x95 fp=0xc8200475a8 sp=0xc820047510
net/smtp.Dial(0xc8200c4c20, 0xc, 0x5badb4, 0x0, 0x0)
    /usr/local/go/src/net/smtp/smtp.go:46 +0x59 fp=0xc820047620 sp=0xc8200475a8
github.com/prometheus/alertmanager/notify.(*Email).Notify(0xc820153930, 0x7fd18888a360, 0xc8204209f0, 0xc820024c58, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:261 +0xa2 fp=0xc820047910 sp=0xc820047620
github.com/prometheus/alertmanager/notify.Build.func1.1(0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:97 +0x155 fp=0xc820047a40 sp=0xc820047910
github.com/prometheus/alertmanager/notify.NotifierFunc.Notify(0xc82016e060, 0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:68 +0x5a fp=0xc820047a80 sp=0xc820047a40
github.com/prometheus/alertmanager/notify.(*RetryNotifier).Notify(0xc820153970, 0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:192 +0x1eb fp=0xc820047c10 sp=0xc820047a80
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc82015ee60, 0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf fp=0xc820047cb8 sp=0xc820047c10
github.com/prometheus/alertmanager/notify.(*DedupingNotifier).Notify(0xc82015ee80, 0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:290 +0x6b2 fp=0xc820047e20 sp=0xc820047cb8
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc82015eea0, 0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf fp=0xc820047ec8 sp=0xc820047e20
github.com/prometheus/alertmanager/notify.Fanout.Notify.func1(0x7fd18888a360, 0xc8204209f0, 0xc820024ac8, 0x1, 0x1, 0xc8203b7c00, 0xc82041ed30, 0x7fd188884898, 0xc82015eea0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:150 +0x86 fp=0xc820047f58 sp=0xc820047ec8
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc820047f60 sp=0xc820047f58
created by github.com/prometheus/alertmanager/notify.Fanout.Notify
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:155 +0x60d

goroutine 1 [chan receive]:
main.main()
    /go/src/github.com/prometheus/alertmanager/main.go:181 +0xf75

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 10 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 72 [IO wait]:
net.runtime_pollWait(0x7fd18888a198, 0x72, 0xc82000c1d0)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202b1aa0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202b1aa0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc8202b1a40, 0x0, 0x7fd18888a290, 0xc8203b7160)
    /usr/local/go/src/net/fd_unix.go:408 +0x27c
net.(*TCPListener).AcceptTCP(0xc820024a40, 0xc820035d70, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
net/http.tcpKeepAliveListener.Accept(0xc820024a40, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:2135 +0x41
net/http.(*Server).Serve(0xc8201a18c0, 0x7fd18888a258, 0xc820024a40, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1887 +0xb3
net/http.(*Server).ListenAndServe(0xc8201a18c0, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1877 +0x136
net/http.ListenAndServe(0x7ffce62fa7c1, 0x5, 0x7fd1888891c0, 0xc82015ef20, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1967 +0x8f
main.listen(0xc82015ef20)
    /go/src/github.com/prometheus/alertmanager/main.go:232 +0x73
created by main.main
    /go/src/github.com/prometheus/alertmanager/main.go:166 +0xcb8

goroutine 19 [select]:
main.(*Dispatcher).run(0xc820070870, 0x7fd188889188, 0xc8203b7120)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:150 +0x8ea
main.(*Dispatcher).Run(0xc820070870)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:59 +0x1de
created by main.main.func3
    /go/src/github.com/prometheus/alertmanager/main.go:152 +0x55a

goroutine 15 [chan receive]:
database/sql.(*DB).connectionOpener(0xc820083900)
    /usr/local/go/src/database/sql/sql.go:634 +0x45
created by database/sql.Open
    /usr/local/go/src/database/sql/sql.go:481 +0x336

goroutine 73 [select, locked to thread]:
runtime.gopark(0xbd5240, 0xc820023f28, 0xae83c0, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820023f28, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820023f28)
    /usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 74 [chan receive]:
main.main.func4(0xc8201a1380, 0xc82006f310)
    /go/src/github.com/prometheus/alertmanager/main.go:176 +0x4d
created by main.main
    /go/src/github.com/prometheus/alertmanager/main.go:179 +0xf52

goroutine 75 [chan receive]:
github.com/prometheus/alertmanager/provider/sqlite.(*Alerts).Subscribe.func1(0xc82000b380, 0x0, 0xc8201a1620, 0xc8203b7100, 0x3, 0x4, 0xc8201a1680)
    /go/src/github.com/prometheus/alertmanager/provider/sqlite/sqlite.go:98 +0x165
created by github.com/prometheus/alertmanager/provider/sqlite.(*Alerts).Subscribe
    /go/src/github.com/prometheus/alertmanager/provider/sqlite/sqlite.go:99 +0x1d8

goroutine 77 [select]:
main.(*aggrGroup).run(0xc820081a00, 0xc82041e530)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:312 +0x1079
created by main.(*Dispatcher).processAlert
    /go/src/github.com/prometheus/alertmanager/dispatch.go:241 +0x42f

goroutine 78 [select]:
main.(*aggrGroup).run(0xc820081a80, 0xc82041e5d0)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:312 +0x1079
created by main.(*Dispatcher).processAlert
    /go/src/github.com/prometheus/alertmanager/dispatch.go:241 +0x42f

goroutine 79 [semacquire]:
sync.runtime_Semacquire(0xc82041ed3c)
    /usr/local/go/src/runtime/sema.go:43 +0x26
sync.(*WaitGroup).Wait(0xc82041ed30)
    /usr/local/go/src/sync/waitgroup.go:126 +0xb4
github.com/prometheus/alertmanager/notify.Fanout.Notify(0xc82016e030, 0x7fd18888a360, 0xc820420780, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:158 +0x63f
github.com/prometheus/alertmanager/notify.Router.Notify(0xc820163fb0, 0x7fd18888a360, 0xc820420780, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:313 +0x28d
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc82015eec0, 0x7fd18888a360, 0xc820420780, 0xc820024ac8, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
github.com/prometheus/alertmanager/notify.(*SilenceNotifier).Notify(0xc82016e1b0, 0x7fd18888a360, 0xc820420780, 0xc820024ab0, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:348 +0x22b
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc82015eee0, 0x7fd18888a360, 0xc820420780, 0xc820024ab0, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
github.com/prometheus/alertmanager/notify.(*InhibitNotifier).Notify(0xc82016e240, 0x7fd18888a360, 0xc820420780, 0xc820024a60, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:383 +0x22b
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc82015ef00, 0x7fd18888a360, 0xc820420780, 0xc820024a60, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
main.(*Dispatcher).processAlert.func1(0x7fd18888a360, 0xc820420780, 0xc820024a60, 0x1, 0x1, 0xc8203b72e0)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:236 +0x8a
main.(*aggrGroup).run.func1(0xc820024a60, 0x1, 0x1, 0x2)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:336 +0x5b
main.(*aggrGroup).flush(0xc820081b00, 0xc8200bfec8)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:400 +0x466
main.(*aggrGroup).run(0xc820081b00, 0xc82041e670)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:337 +0xd53
created by main.(*Dispatcher).processAlert
    /go/src/github.com/prometheus/alertmanager/dispatch.go:241 +0x42f

goroutine 82 [select]:
github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff.(*Ticker).run(0xc82042cc80)
    /go/src/github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff/ticker.go:55 +0x20b
created by github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff.NewTicker
    /go/src/github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff/ticker.go:32 +0x127

Second:

[root@db1 ~]# ./alertmanager-0.1.1.linux-amd64/alertmanager -web.listen-address=:9093 -config.file=./alertmanager.yml -storage.path=./alertmanager-0.1.1.linux-amd64/data
alertmanager, version 0.1.1 (branch: release-0.1, revision: 0e541bf)
  build user:       root@8c44a0677215
  build date:       20160323-10:10:18
  go version:       go1.5.3
INFO[0000] Loading configuration file                    file=./alertmanager.yml source=main.go:125

fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7f58775ef67d]

runtime stack:
runtime.throw(0xb8ef60, 0x2a)
    /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 937 [syscall, locked to thread]:
runtime.cgocall(0x806280, 0xc8204d09d8, 0xc800000000)
    /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc8204d0980 sp=0xc8204d0950
net._C2func_getaddrinfo(0x7f58640008c0, 0x0, 0xc820470480, 0xc820024fc8, 0x0, 0x0, 0x0)
    ??:0 +0x4d fp=0xc8204d09d8 sp=0xc8204d0980
net.cgoLookupIPCNAME(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/cgo_unix.go:111 +0x1ac fp=0xc8204d0b00 sp=0xc8204d09d8
net.cgoLookupIP(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc82000d360)
    /usr/local/go/src/net/cgo_unix.go:163 +0x56 fp=0xc8204d0b58 sp=0xc8204d0b00
net.lookupIP(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup_unix.go:67 +0x94 fp=0xc8204d0bc8 sp=0xc8204d0b58
net.glob.func15(0xbd4d40, 0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/hook.go:10 +0x4d fp=0xc8204d0c08 sp=0xc8204d0bc8
net.lookupIPMerge.func1(0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:68 +0x71 fp=0xc8204d0c88 sp=0xc8204d0c08
internal/singleflight.(*Group).doCall(0x100d770, 0xc820160690, 0xc8200c4c20, 0x9, 0xc8204d0e80)
    /usr/local/go/src/internal/singleflight/singleflight.go:93 +0x2c fp=0xc8204d0d38 sp=0xc8204d0c88
internal/singleflight.(*Group).Do(0x100d770, 0xc8200c4c20, 0x9, 0xc8204d0e80, 0x0, 0x0, 0x0, 0x0, 0xc82000d320)
    /usr/local/go/src/internal/singleflight/singleflight.go:63 +0x284 fp=0xc8204d0da8 sp=0xc8204d0d38
net.lookupIPMerge(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:69 +0x9b fp=0xc8204d0eb8 sp=0xc8204d0da8
net.lookupIPDeadline(0xc8200c4c20, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/lookup.go:91 +0xde fp=0xc8204d10f0 sp=0xc8204d0eb8
net.internetAddrList(0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /usr/local/go/src/net/ipsock.go:252 +0x6ee fp=0xc8204d1258 sp=0xc8204d10f0
net.resolveAddrList(0xae4d60, 0x4, 0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0xc800000000, 0x0, 0x0, ...)
    /usr/local/go/src/net/dial.go:153 +0x466 fp=0xc8204d1330 sp=0xc8204d1258
net.(*Dialer).Dial(0xc8204d1558, 0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/dial.go:209 +0x124 fp=0xc8204d1510 sp=0xc8204d1330
net.Dial(0xae8c38, 0x3, 0xc8200c4c20, 0xc, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/dial.go:186 +0x95 fp=0xc8204d15a8 sp=0xc8204d1510
net/smtp.Dial(0xc8200c4c20, 0xc, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/smtp/smtp.go:46 +0x59 fp=0xc8204d1620 sp=0xc8204d15a8
github.com/prometheus/alertmanager/notify.(*Email).Notify(0xc82014b930, 0x7f58860165c0, 0xc820470300, 0xc820024fc0, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:261 +0xa2 fp=0xc8204d1910 sp=0xc8204d1620
github.com/prometheus/alertmanager/notify.Build.func1.1(0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:97 +0x155 fp=0xc8204d1a40 sp=0xc8204d1910
github.com/prometheus/alertmanager/notify.NotifierFunc.Notify(0xc820166060, 0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/impl.go:68 +0x5a fp=0xc8204d1a80 sp=0xc8204d1a40
github.com/prometheus/alertmanager/notify.(*RetryNotifier).Notify(0xc82014b970, 0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:192 +0x1eb fp=0xc8204d1c10 sp=0xc8204d1a80
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc820152e60, 0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf fp=0xc8204d1cb8 sp=0xc8204d1c10
github.com/prometheus/alertmanager/notify.(*DedupingNotifier).Notify(0xc820152e80, 0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:290 +0x6b2 fp=0xc8204d1e20 sp=0xc8204d1cb8
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc820152ea0, 0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf fp=0xc8204d1ec8 sp=0xc8204d1e20
github.com/prometheus/alertmanager/notify.Fanout.Notify.func1(0x7f58860165c0, 0xc820470300, 0xc820024f98, 0x1, 0x1, 0xc820482900, 0xc82000de00, 0x7f5886010748, 0xc820152ea0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:150 +0x86 fp=0xc8204d1f58 sp=0xc8204d1ec8
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc8204d1f60 sp=0xc8204d1f58
created by github.com/prometheus/alertmanager/notify.Fanout.Notify
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:155 +0x60d

goroutine 1 [chan receive, 59 minutes]:
main.main()
    /go/src/github.com/prometheus/alertmanager/main.go:181 +0xf75

goroutine 17 [syscall, 59 minutes, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 10 [syscall, 59 minutes]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 938 [select]:
github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff.(*Ticker).run(0xc820486940)
    /go/src/github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff/ticker.go:55 +0x20b
created by github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff.NewTicker
    /go/src/github.com/prometheus/alertmanager/vendor/github.com/cenkalti/backoff/ticker.go:32 +0x127

goroutine 15 [chan receive, 59 minutes]:
database/sql.(*DB).connectionOpener(0xc820083900)
    /usr/local/go/src/database/sql/sql.go:634 +0x45
created by database/sql.Open
    /usr/local/go/src/database/sql/sql.go:481 +0x336

goroutine 19 [select]:
main.(*Dispatcher).run(0xc820070870, 0x7f5886010c40, 0xc8201538e0)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:150 +0x8ea
main.(*Dispatcher).Run(0xc820070870)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:59 +0x1de
created by main.main.func3
    /go/src/github.com/prometheus/alertmanager/main.go:152 +0x55a

goroutine 20 [chan receive, 59 minutes]:
github.com/prometheus/alertmanager/provider/sqlite.(*Alerts).Subscribe.func1(0xc82000b380, 0x0, 0xc8201760c0, 0xc8201538c0, 0x3, 0x4, 0xc820176120)
    /go/src/github.com/prometheus/alertmanager/provider/sqlite/sqlite.go:98 +0x165
created by github.com/prometheus/alertmanager/provider/sqlite.(*Alerts).Subscribe
    /go/src/github.com/prometheus/alertmanager/provider/sqlite/sqlite.go:99 +0x1d8

goroutine 77 [IO wait]:
net.runtime_pollWait(0x7f5886016430, 0x72, 0xc82000c1d0)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820291aa0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820291aa0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc820291a40, 0x0, 0x7f5886016528, 0xc82010a760)
    /usr/local/go/src/net/fd_unix.go:408 +0x27c
net.(*TCPListener).AcceptTCP(0xc820024a50, 0x469790, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
net/http.tcpKeepAliveListener.Accept(0xc820024a50, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:2135 +0x41
net/http.(*Server).Serve(0xc820177c20, 0x7f58860164f0, 0xc820024a50, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1887 +0xb3
net/http.(*Server).ListenAndServe(0xc820177c20, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1877 +0x136
net/http.ListenAndServe(0x7ffe9d29f7c1, 0x5, 0x7f5886015458, 0xc820152f20, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1967 +0x8f
main.listen(0xc820152f20)
    /go/src/github.com/prometheus/alertmanager/main.go:232 +0x73
created by main.main
    /go/src/github.com/prometheus/alertmanager/main.go:166 +0xcb8

goroutine 78 [select, 59 minutes, locked to thread]:
runtime.gopark(0xbd5240, 0xc820022f28, 0xae83c0, 0x6, 0x444718, 0x2)
    /usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820022f28, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820022f28)
    /usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 79 [chan receive, 59 minutes]:
main.main.func4(0xc820177a40, 0xc82006f310)
    /go/src/github.com/prometheus/alertmanager/main.go:176 +0x4d
created by main.main
    /go/src/github.com/prometheus/alertmanager/main.go:179 +0xf52

goroutine 918 [select]:
main.(*aggrGroup).run(0xc820468f80, 0xc82044c990)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:312 +0x1079
created by main.(*Dispatcher).processAlert
    /go/src/github.com/prometheus/alertmanager/dispatch.go:241 +0x42f

goroutine 89 [semacquire]:
sync.runtime_Semacquire(0xc82000de0c)
    /usr/local/go/src/runtime/sema.go:43 +0x26
sync.(*WaitGroup).Wait(0xc82000de00)
    /usr/local/go/src/sync/waitgroup.go:126 +0xb4
github.com/prometheus/alertmanager/notify.Fanout.Notify(0xc820166030, 0x7f58860165c0, 0xc8204a3e00, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:158 +0x63f
github.com/prometheus/alertmanager/notify.Router.Notify(0xc820157fb0, 0x7f58860165c0, 0xc8204a3e00, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:313 +0x28d
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc820152ec0, 0x7f58860165c0, 0xc8204a3e00, 0xc820024f98, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
github.com/prometheus/alertmanager/notify.(*SilenceNotifier).Notify(0xc8201661b0, 0x7f58860165c0, 0xc8204a3e00, 0xc8200250f0, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:348 +0x22b
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc820152ee0, 0x7f58860165c0, 0xc8204a3e00, 0xc8200250f0, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
github.com/prometheus/alertmanager/notify.(*InhibitNotifier).Notify(0xc820166240, 0x7f58860165c0, 0xc8204a3e00, 0xc820024f28, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:383 +0x22b
github.com/prometheus/alertmanager/notify.(*LogNotifier).Notify(0xc820152f00, 0x7f58860165c0, 0xc8204a3e00, 0xc820024f28, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/prometheus/alertmanager/notify/notify.go:403 +0x1bf
main.(*Dispatcher).processAlert.func1(0x7f58860165c0, 0xc8204a3e00, 0xc820024f28, 0x1, 0x1, 0xc82000f020)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:236 +0x8a
main.(*aggrGroup).run.func1(0xc820024f28, 0x1, 0x1, 0x2)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:336 +0x5b
main.(*aggrGroup).flush(0xc820468080, 0xc820047ec8)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:400 +0x466
main.(*aggrGroup).run(0xc820468080, 0xc82044ce10)
    /go/src/github.com/prometheus/alertmanager/dispatch.go:337 +0xd53
created by main.(*Dispatcher).processAlert
    /go/src/github.com/prometheus/alertmanager/dispatch.go:241 +0x42f

And so on every time, not sure what's the circumstance leads to this, but I am doing just small things.

brian-brazil · 2016-04-12T16:14:16Z

That looks like a segfault when go is doing DNS lookups

juliusv · 2016-04-12T16:15:30Z

Yeah. Go calls into glibc's getaddrinfo via cgo and it segfaults in there. Something wrong with the glibc maybe? Are you using our binary (completely statically built) version or did you build it yourself?

roman-vynar · 2016-04-12T16:16:57Z

DNS was working fine because Alertmanager was able to send email to the correct email address.

Your tarball https://github.com/prometheus/alertmanager/releases/download/0.1.1/alertmanager-0.1.1.linux-amd64.tar.gz Running on Centos 7.2 x86_64 latest.

juliusv · 2016-04-12T16:20:01Z

@fabxc @sdurrheimer Was this already built with promu or similar? What version of glibc/musl is baked into that?

@roman-vynar While some DNS lookups may work, both crashes strongly indicate that they happened during DNS lookups, in the cgo part (getaddrinfo).

sdurrheimer · 2016-04-12T19:23:06Z

@juliusv This binary looks like it has been build with the prom/golang-builder:1.5.3-main image. This image, at that time, was based on debian:jessie with glibc 2.19-18+deb8u4, but @fabxc should be able to confirm that version if he hasn't pull a updated version since he build alertmanager 0.1.1 with it.

Maybe we should consider always using go internal resolver with the -tags netgo flag.

knyar · 2016-04-21T18:02:20Z

Note, I am seeing similar segfaults with alertmanager built manually from master using make build (built on Debian Wheezy, running on Jessie).

juliusv · 2016-04-21T18:26:14Z

@knyar Interesting, thanks! What Go version are you using? Have you tried the very latest (1.6.2)? It must be either something wrong with the libc, or with Go's cgo handling...

sdurrheimer · 2016-04-21T18:39:47Z

Can you also try to build with the following go build flags to force the use of the Go resolver ?

-a -tags netgo

knyar · 2016-04-21T19:57:43Z

The previous version I was using had been built by 1.5.1 (from Jessie backports package).
I rebuilt master with 1.6.2 (using default build flags), and have not yet been able to trigger a crash.

knyar · 2016-04-23T19:41:27Z

Actually, alertmanager built by 1.6.2 eventually crashed in a similar way.
I've rebuilt with -a -tags netgo, so far so good.

sdurrheimer · 2016-04-29T19:38:06Z

@knyar Did you encountered another crash since you've rebuilt using -a -tags netgo ?

knyar · 2016-04-29T20:08:12Z

No, it's been running for 6 days now without crashing.

sdurrheimer · 2016-04-29T20:23:37Z

@fabxc @juliusv @brian-brazil I think we should put on the table the possibility to always build with the netgo flag, looks like it is more stable than relying on glibc.
It also makes cross-building easier in the most case.

matthiasr · 2016-04-29T20:26:10Z

Agree, netgo will give me consistent results overall and remove this kind of issues.

brian-brazil · 2016-04-29T20:27:07Z

Let's make sure it's clear in the README too, for those building by hand.

matthiasr · 2016-04-29T20:50:31Z

and conversely, anyone who needs name resolution not supported in netgo needs to build by hand.

chbatey · 2016-05-19T09:56:24Z

I'm using the binary from the releases page and getting what looks to be the same issue:

DEBU[1541] Received alert                                alert=Always[8001ae7][active] component=dispatcher source=dispatch.go:16[1476/1963]
DEBU[1551] Received alert                                alert=FalconRunning[51c9220][active] component=dispatcher source=dispatch.go:160
DEBU[1551] Received alert                                alert=Always[8001ae7][active] component=dispatcher source=dispatch.go:160
DEBU[1561] Received alert                                alert=Always[8001ae7][active] component=dispatcher source=dispatch.go:160
DEBU[1561] Received alert                                alert=FalconRunning[51c9220][active] component=dispatcher source=dispatch.go:160
DEBU[1571] Received alert                                alert=Always[8001ae7][active] component=dispatcher source=dispatch.go:160
DEBU[1571] Received alert                                alert=FalconRunning[51c9220][active] component=dispatcher source=dispatch.go:160
DEBU[1581] flushing [FalconRunning[51c9220][active]]     aggrGroup=83abada717c2a0da source=dispatch.go:398
DEBU[1581] notify [FalconRunning[51c9220][active]]       source=notify.go:400 step=inhibit
DEBU[1581] Received alert                                alert=Always[8001ae7][active] component=dispatcher source=dispatch.go:160
DEBU[1581] notify [FalconRunning[51c9220][active]]       source=notify.go:400 step=silence
DEBU[1581] notify [FalconRunning[51c9220][active]]       source=notify.go:400 step=route
DEBU[1581] notify [FalconRunning[51c9220][active]]       source=notify.go:400 step=dedup
DEBU[1581] notify [FalconRunning[51c9220][active]]       source=notify.go:400 step=retry
DEBU[1581] Received alert                                alert=FalconRunning[51c9220][active] component=dispatcher source=dispatch.go:160
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x63 pc=0x7fcde84ea289]

runtime stack:
runtime.throw(0xb8ef60, 0x2a)
        /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.sigpanic()
        /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x5a

goroutine 1023 [syscall, locked to thread]:
runtime.cgocall(0x806280, 0xc820740438, 0xc800000000)
        /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc8207403e0 sp=0xc8207403b0
net._C2func_getaddrinfo(0x7fcdd00008c0, 0x0, 0xc8202f4780, 0xc820036370, 0x0, 0x0, 0x0)
        ??:0 +0x4d fp=0xc820740438 sp=0xc8207403e0
net.cgoLookupIPCNAME(0xc8207a1bc0, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/cgo_unix.go:111 +0x1ac fp=0xc820740560 sp=0xc820740438
net.cgoLookupIP(0xc8207a1bc0, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/cgo_unix.go:163 +0x56 fp=0xc8207405b8 sp=0xc820740560
net.lookupIP(0xc8207a1bc0, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/lookup_unix.go:67 +0x94 fp=0xc820740628 sp=0xc8207405b8
net.glob.func15(0xbd4d40, 0xc8207a1bc0, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/hook.go:10 +0x4d fp=0xc820740668 sp=0xc820740628
net.lookupIPDeadline.func1(0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/lookup.go:106 +0x71 fp=0xc8207406e8 sp=0xc820740668
internal/singleflight.(*Group).doCall(0x100d770, 0xc820847d10, 0xc8207a1bc0, 0xf, 0xc8207a1c80)
        /usr/local/go/src/internal/singleflight/singleflight.go:93 +0x2c fp=0xc820740798 sp=0x

sdurrheimer · 2016-05-19T10:06:58Z

The next release will use the netgo resolver, meanwhile you can build by hand with -a -tags netgo flags.

You can also use a artifact from one of the recent master circleci builds
if this does not bother you : https://circleci.com/gh/prometheus/alertmanager/53#artifacts

chbatey · 2016-05-19T10:21:35Z

Thanks - I will!

sdurrheimer · 2016-07-21T06:58:36Z

@roman-vynar @knyar @chbatey Guys, can we have a final feedback if you are still encountering this kind of problem since we use -a -tags netgo flags ?

roman-vynar · 2016-07-21T07:26:53Z

I have not been using alertmanager for quite some time so I can't provide any update on this.

knyar · 2016-07-21T07:56:45Z

I have not seen this issue on an alertmanager built with -a -tags netgo.
About a week ago I upgraded to the 0.3.0 release binaries, and have had no problems either.

sdurrheimer · 2016-07-22T14:06:33Z

Ok thx for the feedback. I'm gonna close this issue.

- When building static binaries with CGO, cgo unix network segfaults - When building dynamically linked binaries with CGO, everything is fine prometheus/alertmanager#304 golang/go#7857 Signed-off-by: Ying Li <[email protected]>

- When building static binaries with CGO, cgo unix network segfaults - When building dynamically linked binaries with CGO, everything is fine - When building static binaries with CGO but with -tags netgo passed, everything is fine So update our cross and static builds to include netgo. prometheus/alertmanager#304 golang/go#7857 Signed-off-by: Ying Li <[email protected]>

brian-brazil added the bug label Apr 12, 2016

brian-brazil changed the title ~~Crash when notify by email fails~~ Crash in cgo DNS code Apr 12, 2016

fabxc added kind/bug priority/P0 and removed bug labels Apr 27, 2016

sdurrheimer closed this as completed Jul 22, 2016

cyli mentioned this issue Nov 15, 2016

Pass the netgo tag when building static binaries notaryproject/notary#1038

Closed

cyli mentioned this issue Nov 16, 2016

cmd/link: don't swallow external linker warnings golang/go#17935

Closed

TechIsCool mentioned this issue Jan 26, 2018

Is CGO Required for gobetween on Linux yyyar/gobetween#125

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash in cgo DNS code #304

Crash in cgo DNS code #304

roman-vynar commented Apr 12, 2016

juliusv commented Apr 12, 2016

roman-vynar commented Apr 12, 2016

brian-brazil commented Apr 12, 2016

juliusv commented Apr 12, 2016

roman-vynar commented Apr 12, 2016

juliusv commented Apr 12, 2016

sdurrheimer commented Apr 12, 2016

knyar commented Apr 21, 2016

juliusv commented Apr 21, 2016

sdurrheimer commented Apr 21, 2016

knyar commented Apr 21, 2016

knyar commented Apr 23, 2016

sdurrheimer commented Apr 29, 2016

knyar commented Apr 29, 2016

sdurrheimer commented Apr 29, 2016

matthiasr commented Apr 29, 2016 via email

brian-brazil commented Apr 29, 2016

matthiasr commented Apr 29, 2016 via email

chbatey commented May 19, 2016

sdurrheimer commented May 19, 2016

chbatey commented May 19, 2016

sdurrheimer commented Jul 21, 2016

roman-vynar commented Jul 21, 2016 •

edited

Loading

knyar commented Jul 21, 2016

sdurrheimer commented Jul 22, 2016

Crash in cgo DNS code #304

Crash in cgo DNS code #304

Comments

roman-vynar commented Apr 12, 2016

juliusv commented Apr 12, 2016

roman-vynar commented Apr 12, 2016

brian-brazil commented Apr 12, 2016

juliusv commented Apr 12, 2016

roman-vynar commented Apr 12, 2016

juliusv commented Apr 12, 2016

sdurrheimer commented Apr 12, 2016

knyar commented Apr 21, 2016

juliusv commented Apr 21, 2016

sdurrheimer commented Apr 21, 2016

knyar commented Apr 21, 2016

knyar commented Apr 23, 2016

sdurrheimer commented Apr 29, 2016

knyar commented Apr 29, 2016

sdurrheimer commented Apr 29, 2016

matthiasr commented Apr 29, 2016 via email

brian-brazil commented Apr 29, 2016

matthiasr commented Apr 29, 2016 via email

chbatey commented May 19, 2016

sdurrheimer commented May 19, 2016

chbatey commented May 19, 2016

sdurrheimer commented Jul 21, 2016

roman-vynar commented Jul 21, 2016 • edited Loading

knyar commented Jul 21, 2016

sdurrheimer commented Jul 22, 2016

roman-vynar commented Jul 21, 2016 •

edited

Loading