Crash when background fetch fails

[stefanhaller]

Describe the bug
I get a consistent crash when lazygit is running and I set my laptop to sleep. When I open the lid in the morning, lazygit has crashed with the following stack trace:

Crash log

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x28 pc=0x1027d1820]

goroutine 596 [running]:
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).processOutput(0x20?, {0x102c0df00, 0x14000a8e6c0}, {0x102c0e280, 0x140006cc290}, 0x102c08910, {0x0, 0x0})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:334 +0xd0
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).runAndDetectCredentialRequest.func1(0x1400037d740, {0x102c0d760?, 0x1400037d680})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:315 +0x100
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).runAndStreamAux(0x140002f0d60, {0x102c1b068, 0x14000cd8ae0}, 0x140008d7c80)
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:240 +0x538
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).runAndDetectCredentialRequest(0x140002f0d60, {0x102c1b068, 0x14000cd8ae0}, 0x102c08910)
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:312 +0xc0
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).runWithCredentialHandling(0x14000134840?, {0x102c1b068, 0x14000cd8ae0})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:287 +0x54
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*cmdObjRunner).RunWithOutput(0x14000cd8a80?, {0x102c1b068, 0x14000cd8ae0})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj_runner.go:55 +0xf0
github.com/jesseduffield/lazygit/pkg/commands.(*gitCmdObjRunner).RunWithOutput(0x140001402b8, {0x102c1b068, 0x14000cd8a80})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/git_cmd_obj_runner.go:33 +0xe4
github.com/jesseduffield/lazygit/pkg/commands.(*gitCmdObjRunner).Run(0x5?, {0x102c1b068?, 0x14000cd8a80?})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/git_cmd_obj_runner.go:24 +0x24
github.com/jesseduffield/lazygit/pkg/commands/oscommands.(*CmdObj).Run(0x14000126188?)
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/oscommands/cmd_obj.go:185 +0x38
github.com/jesseduffield/lazygit/pkg/commands/git_commands.(*SyncCommands).FetchBackground(0x140007caf08?)
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/commands/git_commands/sync.go:78 +0x28
github.com/jesseduffield/lazygit/pkg/gui.(*BackgroundRoutineMgr).backgroundFetch(0x140002f0d70)
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/gui/background.go:104 +0x30
github.com/jesseduffield/lazygit/pkg/gui.(*BackgroundRoutineMgr).startBackgroundFetch.func1()
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/gui/background.go:64 +0x24
github.com/jesseduffield/lazygit/pkg/gui.(*BackgroundRoutineMgr).goEvery.func1.1({0x140007caf98, 0x1024ad724})
	/Users/stk/Stk/Dev/Builds/lazygit/pkg/gui/background.go:91 +0x2c
github.com/jesseduffield/gocui.(*Gui).onWorkerAux(0x0?, 0x1023725b4?, {0x102c13fb0?, 0x14000a8e5c0?})
	/Users/stk/Stk/Dev/Builds/lazygit/vendor/github.com/jesseduffield/gocui/gui.go:665 +0x6c
github.com/jesseduffield/gocui.(*Gui).OnWorker.func1()
	/Users/stk/Stk/Dev/Builds/lazygit/vendor/github.com/jesseduffield/gocui/gui.go:652 +0x34
created by github.com/jesseduffield/gocui.(*Gui).OnWorker
	/Users/stk/Stk/Dev/Builds/lazygit/vendor/github.com/jesseduffield/gocui/gui.go:651 +0x9c

My suspicion is that MacOS's keychain support for ssh key passphrases is disabled while the lid is closed, so the background fetch fails and is retried, but this crashes because we're cloning the command object, which doesn't clone its task, so the task is nil. This was probably introduced by 16ed3c2.

Note that in earlier versions I would often open the lid to find lazygit not able to fetch or pull. Typing f or p would do nothing. I suspect the root cause might have been the same, just with a different symptom. I never bothered to investigate, so I can't say more about this, sorry. I have to say that the crash is actually an improvement because I can just relaunch lazygit and carry on, whereas previously I would waste a few seconds waiting to see whether anything would happen. 😄

Version info:
Current master: 5cb82a4

[jesseduffield]

The task should be successfully cloned. For example, this test I just wrote passes:

func TestClone(t *testing.T) {
	task := gocui.NewFakeTask()
	cmdObj := &CmdObj{task: task, cmd: &exec.Cmd{}}
	clone := cmdObj.Clone()
	if clone == cmdObj {
		t.Errorf("Clone should not return the same object")
	}

	if clone.GetTask() == nil {
		t.Errorf("Clone task should not be nil")
	}

	if clone.GetTask() != task {
		t.Errorf("Clone should have the same task")
	}
}

So I'm not sure what's going on

[stefanhaller]

You're right, I misread the code thinking that the task isn't cloned.

I ran this under the debugger, and got this, which I find very confusing:

I'm not sure I want to trust the debugger display here, a lot of the variable values just look bogus.

Any other ideas how I could go about diagnosing this?

[stefanhaller]

So the code in processOutput assumes that there's a task when checkForCredentialRequest returns that there's something to do. In my case we get here from backgroundFetch though, and the FetchBackgroundCmdObj doesn't have a task (and presumably shouldn't, because we're called from the long-running background fetch thread).

The credential request in this case is Passphrase, as I suspected. Of course it doesn't make sense to ask the user for a passphrase for a background fetch, so processOutput should probably not be called in this case. A simple if task != nil somewhere could fix it, but I'm not sure if this is the right condition and where exactly in the call stack it should go.

[stefanhaller]

Every morning I'm greeted with crashed lazygit instances when I open my MacBook, so this bothered me enough that I investigated a bit more. It's very easy to reproduce, just remove your ssh key from your ssh agent or whatever you are using to store your pass phrases (in my case, since I use MacOS's keychain, it was just a matter of commenting out the line UseKeychain yes in ~/.ssh/config. Lazygit will then crash on the next background fetch.

Here's a PR that fixes this: #2789

[stefanhaller]

Fixed with #2789.