Skip to content

Add flag to catch and ignore SIGPIPE#6297

Merged
sougou merged 1 commit intovitessio:masterfrom
adsr:catch-sigpipe
Jun 11, 2020
Merged

Add flag to catch and ignore SIGPIPE#6297
sougou merged 1 commit intovitessio:masterfrom
adsr:catch-sigpipe

Conversation

@adsr
Copy link
Copy Markdown
Collaborator

@adsr adsr commented Jun 10, 2020

The Go runtime catches SIGPIPE for us on all fds except stdout and stderr. (See https://golang.org/pkg/os/signal/#hdr-SIGPIPE.) This patch adds a flag that catches SIGPIPE not handled by the runtime, prints a warning, and then ignores all future SIGPIPEs.

In our case, we run Vitess components via systemd. journald was restarted at an unlucky time, causing vttablet to receive a SIGPIPE when trying to write to stderr. As explained above, the Go runtime does not catch SIGPIPE on stdout or stderr, so vttablet was killed.

As an additional note to anyone running systemd services, systemd bizarrely counts death by SIGPIPE (and a few other signals) to be a successful exit. That means you need to set Restart=always if you want your service to restart on SIGPIPE. (We had ours set to Restart=on-failure so the service did not restart.)

Signed-off-by: Adam Saponara as@php.net

Signed-off-by: Adam Saponara <as@php.net>
@adsr adsr requested a review from sougou as a code owner June 10, 2020 16:55
@sougou sougou merged commit ca78ee3 into vitessio:master Jun 11, 2020
@deepthi deepthi added this to the v7.0 milestone Jul 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants