'regs' does not show selected frame registers. #1838

klauspost · 2020-01-19T17:36:14Z

Please answer the following before submitting your issue:

Note: Please include any substantial examples (debug session output,
stacktraces, etc) as linked gists.

What version of Delve are you using (dlv version)?

Latest master.

λ dlv version
Delve Debugger
Version: 1.3.2
Build: $Id: 569ccbd514fc47c8b4c521b142556867ec5e6917 $

What version of Go are you using? (go version)?

go version go1.13.6 windows/amd64

What operating system and processor architecture are you using?

windows/amd64

What did you do?

Reproduction sample: repro.zip

Run the sample. Crash occurs. We switch to the frame causing the crash (3). Print registers.

λ dlv test                                                                                                               
Type 'help' for list of commands.                                                                                        
(dlv) continue                                                                                                           
unexpected fault address 0xc000270398                                                                                    
fatal error: fault                                                                                                       
> [runtime-fatal-throw] runtime.fatalthrow() c:/go/src/runtime/panic.go:820 (hits goroutine(6):1 total:1) (PC: 0x4356f0) 
Warning: debugging optimized function                                                                                    
   815: // fatalthrow implements an unrecoverable runtime throw. It freezes the                                          
   816: // system, prints stack traces starting from its caller, and terminates the                                      
   817: // process.                                                                                                      
   818: //                                                                                                               
   819: //go:nosplit                                                                                                     
=> 820: func fatalthrow() {                                                                                              
   821:         pc := getcallerpc()                                                                                      
   822:         sp := getcallersp()                                                                                      
   823:         gp := getg()                                                                                             
   824:         // Switch to the system stack to avoid any stack growth, which                                           
   825:         // may make things worse if the runtime is in a bad state.                                               
(dlv) stack                                                                                                              
0  0x00000000004356f0 in runtime.fatalthrow                                                                              
   at c:/go/src/runtime/panic.go:820                                                                                     
1  0x0000000000435579 in runtime.throw                                                                                   
   at c:/go/src/runtime/panic.go:774                                                                                     
2  0x0000000000448463 in runtime.sigpanic                                                                                
   at c:/go/src/runtime/signal_windows.go:236                                                                            
3  0x0000000000560598 in _/e_/temp/weird.repro                                                                           
   at e:/temp/weird/repro.s:6                                                                                            
4  0x0000000000560579 in _/e_/temp/weird.TestRepro                                                                       
   at e:/temp/weird/test_test.go:6                                                                                       
5  0x000000000050a353 in testing.tRunner                                                                                 
   at c:/go/src/testing/testing.go:909                                                                                   
6  0x0000000000463c71 in runtime.goexit                                                                                  
   at c:/go/src/runtime/asm_amd64.s:1357                                                                                 
(dlv) frame 3                                                                                                            
> [runtime-fatal-throw] runtime.fatalthrow() c:/go/src/runtime/panic.go:820 (hits goroutine(6):1 total:1) (PC: 0x4356f0) 
Warning: debugging optimized function                                                                                    
Frame 3: e:/temp/weird/repro.s:6 (PC: 560598)                                                                            
     1: #include "textflag.h"                                                                                            
     2:                                                                                                                  
     3: // func repro(src []byte)                                                                                        
     4: TEXT ·repro(SB), NOSPLIT, $0-24                                                                                  
     5:         XORQ AX, AX                                                                                              
=>   6:         LEAQ src_base+0(FP), CX                                                                                  
     7:         MOVQ AX, 2000000(CX)                                                                                     
     8:         RET                                                                                                      
(dlv) disassemble
TEXT _/e_/temp/weird.repro(SB) e:/temp/weird/repro.s
        repro.s:5       0x560590        4831c0          xor rax, rax
        repro.s:6       0x560593        488d4c2408      lea rcx, ptr [rsp+0x8]
        repro.s:7       0x560598        48898180841e00  mov qword ptr [rcx+0x1e8480], rax
        repro.s:8       0x56059f        c3              ret
(dlv) regs                                                                                                               
   Rip = 0x00000000004356f0                                                                                              
   Rsp = 0x000000c000087ea8                                                                                              
   Rax = 0x00000000006bc5e0                                                                                              
   Rbx = 0x00000000006bc5e0                                                                                              
   Rcx = 0x00000000006bc638                                                                                              
   Rdx = 0x00000000006db090                                                                                              
   Rdi = 0x0000000000258000                                                                                              
   Rsi = 0x000000000099fd01                                                                                              
   Rbp = 0x000000c000087ed0                                                                                              
    R8 = 0x000000000099fbc8                                                                                              
    R9 = 0x000000000099fd48                                                                                              
   R10 = 0x0000000000000000                                                                                              
   R11 = 0x0000000000000246                                                                                              
   R12 = 0x0000000002030000                                                                                              
   R13 = 0x0000000000000000                                                                                              
   R14 = 0x00000000000000d0                                                                                              
   R15 = 0x0000000000000000                                                                                              
Eflags = 0x0000000000000244     [PF ZF IF IOPL=0]                                                                        
    Cs = 0x0000000000000033                                                                                              
    Fs = 0x0000000000000053                                                                                              
    Gs = 0x000000000000002b                                                                                              
   TLS = 0x0000000000258000                                                                                              
(dlv)

What did you expect to see?

RAX to be 0. Note the XORQ AX, AX before the crash was invoked.

Also the instruction pointed to is wrong. There is AFAIK no way for LEAQ to fail.

What did you see instead?

RAX shows the registers at frame 0 as far as I can tell.

Thanks for a great tool!

The text was updated successfully, but these errors were encountered:

chainhelen · 2020-01-20T04:23:52Z

@klauspost This is not problem of dlv.
It is beacause when process panic if invalid operations(ex: access invalid memory), process will received a signal (SIGSEGV) from os. Then process (golang) will enter runtime.sigpanic to handle this exception. This affect the values of regs.

There is AFAIK no way for LEAQ to fail.

I test it. It is because cpu execute repro.s:7 failed.
And the flow from repro.s:6 enter runtime.sigpanic.
So not wrong with LEAQ but MOVQ AX, 2000000(CX).

(dlv) n
> _/C_/Users/admin/Desktop/repro.repro() C:/Users/admin/Desktop/repro/repro.s:6 (PC: 0x560593)
     1: #include "textflag.h"
     2:
     3: // func repro(src []byte)
     4: TEXT ·repro(SB), NOSPLIT, $0-24
     5:         XORQ AX, AX
=>   6:         LEAQ src_base+0(FP), CX
     7:         MOVQ AX, 2000000(CX)
     8:         RET
(dlv) n
> _/C_/Users/admin/Desktop/repro.repro() C:/Users/admin/Desktop/repro/repro.s:7 (PC: 0x560598)
     2:
     3: // func repro(src []byte)
     4: TEXT ·repro(SB), NOSPLIT, $0-24
     5:         XORQ AX, AX
     6:         LEAQ src_base+0(FP), CX
=>   7:         MOVQ AX, 2000000(CX)
     8:         RET
(dlv) n
unexpected fault address 0xc000230398
fatal error: fault
> [runtime-fatal-throw] runtime.fatalthrow() c:/go/src/runtime/panic.go:820 (hits goroutine(19):1 total:1) (PC: 0x4356f0)
Warning: debugging optimized function
   815: // fatalthrow implements an unrecoverable runtime throw. It freezes the
   816: // system, prints stack traces starting from its caller, and terminates the
   817: // process.
   818: //
   819: //go:nosplit
=> 820: func fatalthrow() {
   821:         pc := getcallerpc()
   822:         sp := getcallersp()
   823:         gp := getg()
   824:         // Switch to the system stack to avoid any stack growth, which
   825:         // may make things worse if the runtime is in a bad state.

aarzilli · 2020-01-20T08:36:59Z

In pratice most frames won't have any registers except the stack and frame pointers. This frame would be an exception because of the context pushed on the stack for the signal handler, but we don't parse that at the moment.

klauspost · 2020-01-20T09:27:48Z

[...] context pushed on the stack for the signal handler, but we don't parse that at the moment.

@aarzilli That is probably what I was expecting. All other debuggers I have worked with will display registers at any given frame. Obviously, this makes assembly debugging quite hard since registers will have been overwritten with random data.

In pratice most frames won't have any registers except the stack and frame pointers.

That sentence doesn't make sense to me. Most Go code uses registers.

I test it. It is because cpu execute repro.s:7 failed.
So not wrong with LEAQ but MOVQ AX, 2000000(CX).

@chainhelen Yes, that is my point. If it doesn't fail, just increase 2000000. My point is also that this is still reported wrong above, and you don't address the main issue.

aarzilli · 2020-01-20T09:36:44Z

In pratice most frames won't have any registers except the stack and frame pointers.

That sentence doesn't make sense to me. Most Go code uses registers.

Go doesn't track where registers are saved on the stack (besides the stack pointer) so most frames will not have any registers.

klauspost · 2020-01-20T10:04:30Z

@aarzilli Ah, now I get what you mean. I do still think it is quite important to reliably see what registers were when a crash occurs.

klauspost · 2020-01-21T10:12:14Z

Example:

λ dlv test
Type 'help' for list of commands.
(dlv) c
unexpected fault address 0xc0110fe0f0
fatal error: fault
> [runtime-fatal-throw] runtime.fatalthrow() c:/go/src/runtime/panic.go:820 (hits goroutine(544):1 total:1) (PC: 0x436d90)
Warning: debugging optimized function
   815: // fatalthrow implements an unrecoverable runtime throw. It freezes the
   816: // system, prints stack traces starting from its caller, and terminates the
   817: // process.
   818: //
   819: //go:nosplit
=> 820: func fatalthrow() {
   821:         pc := getcallerpc()
   822:         sp := getcallersp()
   823:         gp := getg()
   824:         // Switch to the system stack to avoid any stack growth, which
   825:         // may make things worse if the runtime is in a bad state.
(dlv) frame 3
> [runtime-fatal-throw] runtime.fatalthrow() c:/go/src/runtime/panic.go:820 (hits goroutine(544):1 total:1) (PC: 0x436d90)
Warning: debugging optimized function
Frame 3: c:/gopath/src/github.com/klauspost/compress/s2/encodeblock_amd64.s:638 (PC: 7d8ebb)
   633:
   634: no_repeat_found_encodeBlockAsm:
   635:         CMPL (CX)(BX*1), BP
   636:         SHRQ $0x08, BP
   637:         JEQ  candidate_match_encodeBlockAsm
=> 638:         MOVL (SP)(R8*1), BX
   639:         CMPL (CX)(SI*1), BP
   640:         JEQ  candidate2_match_encodeBlockAsm
   641:         LEAL 2(AX), SI
   642:         MOVL SI, (SP)(R8*1)
   643:         SHRQ $0x08, BP
(dlv) regs
   Rip = 0x0000000000436d90
   Rsp = 0x000000c00024d880
   Rax = 0x000000c000044e00
   Rbx = 0x000000c000044e00
   Rcx = 0x000000c000044e58
   Rdx = 0x0000000000b3e828
   Rdi = 0x00000000002f7000
   Rsi = 0x000000000317fd01
   Rbp = 0x000000c00024d8a8
    R8 = 0x000000000317fbf8
    R9 = 0x000000000317fd70
   R10 = 0x0000000000000000
   R11 = 0x0000000000000246
   R12 = 0x0000000000000276
   R13 = 0x000000c000112280
   R14 = 0x0000000000000006
   R15 = 0x0000000000000000
Eflags = 0x0000000000000244     [PF ZF IF IOPL=0]
    Cs = 0x0000000000000033
    Fs = 0x0000000000000053
    Gs = 0x000000000000002b
   TLS = 0x00000000002f7000

(dlv)

I have a crash, but I am unable to see the values in the registers at the time of the crash. So I basically have no information available.

Also the crash is actually at the line below (CMPL)

heschi · 2020-01-23T05:12:55Z

I'd think this would be generally useful for panics, especially in optimized binaries with location lists.

I think @aarzilli pointed to this, but to be explicit: It should be possible to recover the registers in at least some circumstances. On Linux it'd be the ctxt argument to sigtrampgo: https://github.com/golang/go/blob/67f0f83216930e053441500e2b28c3fa2b667581/src/runtime/signal_unix.go#L403 On Windows it looks like https://github.com/golang/go/blob/6dbcc8b8651909442ff823231daba096f447a163/src/runtime/signal_windows.go#L104 messes with things badly enough that it would be hard to do.

It seems easy enough to add a sigctxt field somewhere (maybe the m?) and have all the OS handlers fill that in with whatever the platform-specific context structure is. Please feel free to file an issue if that'd help.

A significant amount of time is spent generating the string representation for the proc.Registers object of each thread, since this field is rarely used (only when the Registers API is called) it should be generated on demand. Also by changing the internal representation of proc.Register to be closer to that of op.DwarfRegister it will help us implement go-delve#1838 (when Delve will need to be able to display the registers of an internal frame, which we currently represent using op.DwarfRegister objects). Benchmark before: BenchmarkConditionalBreakpoints-4 1 22292554301 ns/op Benchmark after: BenchmarkConditionalBreakpoints-4 1 17326345671 ns/op Reduces conditional breakpoint latency from 2.2ms to 1.7ms. Updates go-delve#1549, go-delve#1838

A significant amount of time is spent generating the string representation for the proc.Registers object of each thread, since this field is rarely used (only when the Registers API is called) it should be generated on demand. Also by changing the internal representation of proc.Register to be closer to that of op.DwarfRegister it will help us implement #1838 (when Delve will need to be able to display the registers of an internal frame, which we currently represent using op.DwarfRegister objects). Benchmark before: BenchmarkConditionalBreakpoints-4 1 22292554301 ns/op Benchmark after: BenchmarkConditionalBreakpoints-4 1 17326345671 ns/op Reduces conditional breakpoint latency from 2.2ms to 1.7ms. Updates #1549, #1838

Adds an optional scope prefix to the `regs` command which allows printing registers for any stack frame (as long as they were somehow saved). Issue go-delve#1838 is not yet to be closed since we are still not recovering the registers of a segfaulting frame. Updates go-delve#1838

Adds an optional scope prefix to the `regs` command which allows printing registers for any stack frame (as long as they were somehow saved). Issue #1838 is not yet to be closed since we are still not recovering the registers of a segfaulting frame. Updates #1838

A significant amount of time is spent generating the string representation for the proc.Registers object of each thread, since this field is rarely used (only when the Registers API is called) it should be generated on demand. Also by changing the internal representation of proc.Register to be closer to that of op.DwarfRegister it will help us implement go-delve#1838 (when Delve will need to be able to display the registers of an internal frame, which we currently represent using op.DwarfRegister objects). Benchmark before: BenchmarkConditionalBreakpoints-4 1 22292554301 ns/op Benchmark after: BenchmarkConditionalBreakpoints-4 1 17326345671 ns/op Reduces conditional breakpoint latency from 2.2ms to 1.7ms. Updates go-delve#1549, go-delve#1838

…#1875) Adds an optional scope prefix to the `regs` command which allows printing registers for any stack frame (as long as they were somehow saved). Issue go-delve#1838 is not yet to be closed since we are still not recovering the registers of a segfaulting frame. Updates go-delve#1838

A significant amount of time is spent generating the string representation for the proc.Registers object of each thread, since this field is rarely used (only when the Registers API is called) it should be generated on demand. Also by changing the internal representation of proc.Register to be closer to that of op.DwarfRegister it will help us implement go-delve#1838 (when Delve will need to be able to display the registers of an internal frame, which we currently represent using op.DwarfRegister objects). Benchmark before: BenchmarkConditionalBreakpoints-4 1 22292554301 ns/op Benchmark after: BenchmarkConditionalBreakpoints-4 1 17326345671 ns/op Reduces conditional breakpoint latency from 2.2ms to 1.7ms. Updates go-delve#1549, go-delve#1838

…#1875) Adds an optional scope prefix to the `regs` command which allows printing registers for any stack frame (as long as they were somehow saved). Issue go-delve#1838 is not yet to be closed since we are still not recovering the registers of a segfaulting frame. Updates go-delve#1838

aarzilli added kind/enhancement area/proc area/proc/native labels Jan 20, 2020

aarzilli mentioned this issue Feb 12, 2020

proc: only format registers value when it's necessary #1860

Merged

aarzilli mentioned this issue Mar 5, 2020

runtime: save CPU registers for the frame calling runtime.sigpanic golang/go#37683

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'regs' does not show selected frame registers. #1838

'regs' does not show selected frame registers. #1838

klauspost commented Jan 19, 2020 •

edited

Loading

chainhelen commented Jan 20, 2020 •

edited

Loading

aarzilli commented Jan 20, 2020

klauspost commented Jan 20, 2020

aarzilli commented Jan 20, 2020

klauspost commented Jan 20, 2020

klauspost commented Jan 21, 2020 •

edited

Loading

heschi commented Jan 23, 2020

'regs' does not show selected frame registers. #1838

'regs' does not show selected frame registers. #1838

Comments

klauspost commented Jan 19, 2020 • edited Loading

chainhelen commented Jan 20, 2020 • edited Loading

aarzilli commented Jan 20, 2020

klauspost commented Jan 20, 2020

aarzilli commented Jan 20, 2020

klauspost commented Jan 20, 2020

klauspost commented Jan 21, 2020 • edited Loading

heschi commented Jan 23, 2020

klauspost commented Jan 19, 2020 •

edited

Loading

chainhelen commented Jan 20, 2020 •

edited

Loading

klauspost commented Jan 21, 2020 •

edited

Loading