Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fail to get stack frame on crash 7.2.0 #22

Open
kasks80 opened this issue Oct 24, 2017 · 10 comments
Open

fail to get stack frame on crash 7.2.0 #22

kasks80 opened this issue Oct 24, 2017 · 10 comments

Comments

@kasks80
Copy link

kasks80 commented Oct 24, 2017

when trying to get stack frame with the command such as "bt -f" then, I could see a warning message like below
"cannot determine starting stack frame for task".

and still could not get the correct callstack with its registers even though message was just a warning.

could anyone solve this problem??

Thanks,

@crash-utility
Copy link
Collaborator

crash-utility commented Oct 24, 2017 via email

@kasks80
Copy link
Author

kasks80 commented Oct 25, 2017

What kind of dumpfile is it? What architecture?
-> arch is arm64.(current I'm developing the android project)
-> how can i check kind of dumpfile?

I was confused. I mean when using "bt -t" or "bt -a", I could see below things.

PID: 16645 TASK: ffffffc027fa9180 CPU: 0 COMMAND: "Binder:16621_1"
bt: WARNING: cannot determine starting stack frame for task ffffffc027fa9180

actually I expected like the below result.(copy example from "help bt")

crash> bt -a
PID: 286    TASK: c0b3a000  CPU: 0   COMMAND: "in.rlogind"
#0 [c0b3be90] crash_save_current_state at c011aed0
#1 [c0b3bea4] panic at c011367c
#2 [c0b3bee8] tulip_interrupt at c01bc820
#3 [c0b3bf08] handle_IRQ_event at c010a551
#4 [c0b3bf2c] do_8259A_IRQ at c010a319
#5 [c0b3bf3c] do_IRQ at c010a653
#6 [c0b3bfbc] ret_from_intr at c0109634
   EAX: 00000000  EBX: c0e68280  ECX: 00000000  EDX: 00000004  EBP: c0b3bfbc
   DS:  0018      ESI: 00000004  ES:  0018      EDI: c0e68284
   CS:  0010      EIP: c012f803  ERR: ffffff09  EFLAGS: 00000246
#7 [c0b3bfbc] sys_select at c012f803
#8 [c0b3bfc0] system_call at c0109598
   EAX: 0000008e  EBX: 00000004  ECX: bfffc9a0  EDX: 00000000
   DS:  002b      ESI: bfffc8a0  ES:  002b      EDI: 00000000
   SS:  002b      ESP: bfffc82c  EBP: bfffd224
   CS:  0023      EIP: 400d032e  ERR: 0000008e  EFLAGS: 00000246

@crash-utility
Copy link
Collaborator

crash-utility commented Oct 25, 2017 via email

@8yanghao
Copy link

I also meet such problem. on ARM64 platform I also can't restore the task bt;
crash> help -m
... [ cut ] ...
panic_task_regs: 0
...
but actually I get restore the pc, sp ptr from kmsg print or other way, is there any way to set the x0~x31 and PC, SP ptr by hand? and then the bt can work fine?

@crash-utility
Copy link
Collaborator

crash-utility commented Apr 23, 2019 via email

@slowpy
Copy link

slowpy commented Jul 9, 2020

in arm64,the crash tool check pstate

@taigerhu
Copy link

taigerhu commented Oct 28, 2020

I also meet such problem. on ARM64 platform I also can't restore the task bt;
crash> help -m
... [ cut ] ...
panic_task_regs: 0
...
but actually I get restore the pc, sp ptr from kmsg print or other way, is there any way to set the x0~x31 and PC, SP ptr by hand? and then the bt can work fine?

hi

i already debug the function, seems no crash_notes symbol, did you side fix the issue?

Thx.

@praton1729
Copy link

praton1729 commented Aug 13, 2021

Hi guys,

I was able to get the bt(still with one error message) by adding/subtracting(yes, both worked) 8 from sp on arm64. The printed register values are not correct though.

image

P.S: help -D and help -m output is attached.
help_D.txt
help_m.txt

@YuanyeMa
Copy link

YuanyeMa commented Sep 1, 2022

Hi guys,
I also meet such problem. on ARM64 platform.
Adding/subtracting 8 from sp is does not work yet.

Corefile type : vmcore_data
panic_task_regs: 0

crash> bt 1
PID: 1 TASK: ffffff95c514bd40 CPU: 7 COMMAND: "init"
bt: WARNING: cannot determine starting stack frame for task ffffff95c514bd40
crash> bt -I 0x00000558f52a1a4 -S 0x000007fca225500 1
PID: 1 TASK: ffffff95c514bd40 CPU: 7 COMMAND: "init"
bt: non-process stack address for this task: 7fca225500
(valid range: ffffffc010060000 - ffffffc010064000)
crash> bt -I 0x00000558f52a1a4 -S 0x000007fca225508 1
PID: 1 TASK: ffffff95c514bd40 CPU: 7 COMMAND: "init"
bt: non-process stack address for this task: 7fca225508
(valid range: ffffffc010060000 - ffffffc010064000)

Does anyone have any ideas to solve this problem?

Thanks Very Much

@YuanyeMa
Copy link

YuanyeMa commented Sep 2, 2022

When the crash is just started, the following print is displayed. Does it matter?

WARNING: cpu 0: cannot find NT_PRSTATUS note
WARNING: cpu 1: cannot find NT_PRSTATUS note
WARNING: cpu 2: cannot find NT_PRSTATUS note
WARNING: cpu 3: cannot find NT_PRSTATUS note
WARNING: cpu 4: cannot find NT_PRSTATUS note
WARNING: cpu 5: cannot find NT_PRSTATUS note
WARNING: cpu 6: cannot find NT_PRSTATUS note
WARNING: cpu 7: cannot find NT_PRSTATUS note

Also, is the issue related to zram enabled in the kernel?

Thanks.

k-hagio pushed a commit that referenced this issue Nov 29, 2023
…usly

There is an issue that, for kernel modules, "dis -rl" fails to display
modules code line number data after execute "bt" command in crash.

Without the patch:
  crsah> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   #8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp
  0xffffffffc0f60eb6 <lpfc_nlp_get+6>:    push   %rbx
  0xffffffffc0f60eb7 <lpfc_nlp_get+7>:    test   %rdi,%rdi

With the patch:
  crash> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   #8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6756
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6759
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp

The root cause is, after kernel module been loaded by mod command, the symtable
is not expanded in gdb side. crash bt or dis command will trigger such an
expansion. However the symtable expansion is different for the 2 commands:

The stack trace of "dis -rl" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  #1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  #2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  #3  0x000000000077e8e9 in process_full_comp_unit ...
  #4  process_queue ...
  #5  dw2_do_instantiate_symtab ...
  #6  0x000000000077ed67 in dw2_instantiate_symtab ...
  #7  0x000000000077f75e in dw2_expand_all_symtabs ...
  #8  0x00000000008f254d in gdb_get_line_number ...
  #9  0x00000000008f22af in gdb_command_funnel_1 ...
  #10 0x00000000008f2003 in gdb_command_funnel ...
  #11 0x00000000005b7f02 in gdb_interface ...
  #12 0x00000000005f5bd8 in get_line_number ...
  #13 0x000000000059e574 in cmd_dis ...

The stack trace of "bt" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  #1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  #2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  #3  0x000000000077e8e9 in process_full_comp_unit ...
  #4  process_queue ...
  #5  dw2_do_instantiate_symtab ...
  #6  0x000000000077ed67 in dw2_instantiate_symtab ...
  #7  0x000000000077f8ed in dw2_lookup_symbol ...
  #8  0x00000000008e6d03 in lookup_symbol_via_quick_fns ...
  #9  0x00000000008e7153 in lookup_symbol_in_objfile ...
  #10 0x00000000008e73c6 in lookup_symbol_global_or_static_iterator_cb ...
  #11 0x00000000008b99c4 in svr4_iterate_over_objfiles_in_search_order ...
  #12 0x00000000008e754e in lookup_global_or_static_symbol ...
  #13 0x00000000008e75da in lookup_static_symbol ...
  #14 0x00000000008e632c in lookup_symbol_aux ...
  #15 0x00000000008e5a7a in lookup_symbol_in_language ...
  #16 0x00000000008e5b30 in lookup_symbol ...
  #17 0x00000000008f2a4a in gdb_get_datatype ...
  #18 0x00000000008f22c0 in gdb_command_funnel_1 ...
  #19 0x00000000008f2003 in gdb_command_funnel ...
  #20 0x00000000005b7f02 in gdb_interface ...
  #21 0x00000000005f8a9f in datatype_info ...
  #22 0x0000000000599947 in cpu_map_size ...
  #23 0x00000000005a975d in get_cpus_online ...
  #24 0x0000000000637a8b in diskdump_get_prstatus_percpu ...
  #25 0x000000000062f0e4 in get_netdump_regs_x86_64 ...
  #26 0x000000000059fe68 in back_trace ...
  #27 0x00000000005ab1cb in cmd_bt ...

For the stacktrace of "dis -rl", it calls dw2_expand_all_symtabs() to expand
all symtable of the objfile, or "*.ko.debug" in our case. However for
the stacktrace of "bt", it doesn't expand all, but only a subset of symtable
which is enough to find a symbol by dw2_lookup_symbol(). As a result, the
objfile->compunit_symtabs, which is the head of a single linked list of
struct compunit_symtab, is not NULL but didn't contain all symtables. It
will not be reinitialized in gdb_get_line_number() by "dis -rl" because
!objfile_has_full_symbols(objfile) check will fail, so it cannot display
the proper code line number data.

Since objfile_has_full_symbols(objfile) check cannot ensure all symbols
been expanded, this patch add a new member as a flag for struct objfile
to record if all symbols have been expanded. The flag will be set only ofter
expand_all_symtabs been called.

Signed-off-by: Tao Liu <[email protected]>
liutgnu added a commit to liutgnu/crash-preview that referenced this issue Nov 30, 2023
…usly

There is an issue that, for kernel modules, "dis -rl" fails to display
modules code line number data after execute "bt" command in crash.

Without the patch:
  crsah> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp
  0xffffffffc0f60eb6 <lpfc_nlp_get+6>:    push   %rbx
  0xffffffffc0f60eb7 <lpfc_nlp_get+7>:    test   %rdi,%rdi

With the patch:
  crash> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6756
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6759
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp

The root cause is, after kernel module been loaded by mod command, the symtable
is not expanded in gdb side. crash bt or dis command will trigger such an
expansion. However the symtable expansion is different for the 2 commands:

The stack trace of "dis -rl" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f75e in dw2_expand_all_symtabs ...
  crash-utility#8  0x00000000008f254d in gdb_get_line_number ...
  crash-utility#9  0x00000000008f22af in gdb_command_funnel_1 ...
  crash-utility#10 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#11 0x00000000005b7f02 in gdb_interface ...
  crash-utility#12 0x00000000005f5bd8 in get_line_number ...
  crash-utility#13 0x000000000059e574 in cmd_dis ...

The stack trace of "bt" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f8ed in dw2_lookup_symbol ...
  crash-utility#8  0x00000000008e6d03 in lookup_symbol_via_quick_fns ...
  crash-utility#9  0x00000000008e7153 in lookup_symbol_in_objfile ...
  crash-utility#10 0x00000000008e73c6 in lookup_symbol_global_or_static_iterator_cb ...
  crash-utility#11 0x00000000008b99c4 in svr4_iterate_over_objfiles_in_search_order ...
  crash-utility#12 0x00000000008e754e in lookup_global_or_static_symbol ...
  crash-utility#13 0x00000000008e75da in lookup_static_symbol ...
  crash-utility#14 0x00000000008e632c in lookup_symbol_aux ...
  crash-utility#15 0x00000000008e5a7a in lookup_symbol_in_language ...
  crash-utility#16 0x00000000008e5b30 in lookup_symbol ...
  crash-utility#17 0x00000000008f2a4a in gdb_get_datatype ...
  crash-utility#18 0x00000000008f22c0 in gdb_command_funnel_1 ...
  crash-utility#19 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#20 0x00000000005b7f02 in gdb_interface ...
  crash-utility#21 0x00000000005f8a9f in datatype_info ...
  crash-utility#22 0x0000000000599947 in cpu_map_size ...
  crash-utility#23 0x00000000005a975d in get_cpus_online ...
  crash-utility#24 0x0000000000637a8b in diskdump_get_prstatus_percpu ...
  crash-utility#25 0x000000000062f0e4 in get_netdump_regs_x86_64 ...
  crash-utility#26 0x000000000059fe68 in back_trace ...
  crash-utility#27 0x00000000005ab1cb in cmd_bt ...

For the stacktrace of "dis -rl", it calls dw2_expand_all_symtabs() to expand
all symtable of the objfile, or "*.ko.debug" in our case. However for
the stacktrace of "bt", it doesn't expand all, but only a subset of symtable
which is enough to find a symbol by dw2_lookup_symbol(). As a result, the
objfile->compunit_symtabs, which is the head of a single linked list of
struct compunit_symtab, is not NULL but didn't contain all symtables. It
will not be reinitialized in gdb_get_line_number() by "dis -rl" because
!objfile_has_full_symbols(objfile) check will fail, so it cannot display
the proper code line number data.

Since objfile_has_full_symbols(objfile) check cannot ensure all symbols
been expanded, this patch add a new member as a flag for struct objfile
to record if all symbols have been expanded. The flag will be set only ofter
expand_all_symtabs been called.

Signed-off-by: Tao Liu <[email protected]>
liutgnu added a commit to liutgnu/crash-preview that referenced this issue Dec 1, 2023
…eviously

There is an issue that, for kernel modules, "dis -rl" fails to display
modules code line number data after execute "bt" command in crash.

Without the patch:
  crsah> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp
  0xffffffffc0f60eb6 <lpfc_nlp_get+6>:    push   %rbx
  0xffffffffc0f60eb7 <lpfc_nlp_get+7>:    test   %rdi,%rdi

With the patch:
  crash> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6756
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6759
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp

The root cause is, after kernel module been loaded by mod command, the symtable
is not expanded in gdb side. crash bt or dis command will trigger such an
expansion. However the symtable expansion is different for the 2 commands:

The stack trace of "dis -rl" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f75e in dw2_expand_all_symtabs ...
  crash-utility#8  0x00000000008f254d in gdb_get_line_number ...
  crash-utility#9  0x00000000008f22af in gdb_command_funnel_1 ...
  crash-utility#10 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#11 0x00000000005b7f02 in gdb_interface ...
  crash-utility#12 0x00000000005f5bd8 in get_line_number ...
  crash-utility#13 0x000000000059e574 in cmd_dis ...

The stack trace of "bt" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f8ed in dw2_lookup_symbol ...
  crash-utility#8  0x00000000008e6d03 in lookup_symbol_via_quick_fns ...
  crash-utility#9  0x00000000008e7153 in lookup_symbol_in_objfile ...
  crash-utility#10 0x00000000008e73c6 in lookup_symbol_global_or_static_iterator_cb ...
  crash-utility#11 0x00000000008b99c4 in svr4_iterate_over_objfiles_in_search_order ...
  crash-utility#12 0x00000000008e754e in lookup_global_or_static_symbol ...
  crash-utility#13 0x00000000008e75da in lookup_static_symbol ...
  crash-utility#14 0x00000000008e632c in lookup_symbol_aux ...
  crash-utility#15 0x00000000008e5a7a in lookup_symbol_in_language ...
  crash-utility#16 0x00000000008e5b30 in lookup_symbol ...
  crash-utility#17 0x00000000008f2a4a in gdb_get_datatype ...
  crash-utility#18 0x00000000008f22c0 in gdb_command_funnel_1 ...
  crash-utility#19 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#20 0x00000000005b7f02 in gdb_interface ...
  crash-utility#21 0x00000000005f8a9f in datatype_info ...
  crash-utility#22 0x0000000000599947 in cpu_map_size ...
  crash-utility#23 0x00000000005a975d in get_cpus_online ...
  crash-utility#24 0x0000000000637a8b in diskdump_get_prstatus_percpu ...
  crash-utility#25 0x000000000062f0e4 in get_netdump_regs_x86_64 ...
  crash-utility#26 0x000000000059fe68 in back_trace ...
  crash-utility#27 0x00000000005ab1cb in cmd_bt ...

For the stacktrace of "dis -rl", it calls dw2_expand_all_symtabs() to expand
all symtable of the objfile, or "*.ko.debug" in our case. However for
the stacktrace of "bt", it doesn't expand all, but only a subset of symtable
which is enough to find a symbol by dw2_lookup_symbol(). As a result, the
objfile->compunit_symtabs, which is the head of a single linked list of
struct compunit_symtab, is not NULL but didn't contain all symtables. It
will not be reinitialized in gdb_get_line_number() by "dis -rl" because
!objfile_has_full_symbols(objfile) check will fail, so it cannot display
the proper code line number data.

Since objfile_has_full_symbols(objfile) check cannot ensure all symbols
been expanded, this patch add a new member as a flag for struct objfile
to record if all symbols have been expanded. The flag will be set only ofter
expand_all_symtabs been called.

Signed-off-by: Tao Liu <[email protected]>
sugarfillet pushed a commit to sugarfillet/crash that referenced this issue Dec 13, 2023
The patch introduces per-cpu overflow stacks for RISCV64 to let
"bt" do backtrace on it and the 'help -m' command dispalys the
addresss of each per-cpu overflow stack.

TEST: a lkdtm DIRECT EXHAUST_STACK vmcore

```
crash> bt
PID: 1        TASK: ff600000000d8000  CPU: 1    COMMAND: "sh"
 #0 [ff6000001fc501c0] riscv_crash_save_regs at ffffffff8000a1dc
 crash-utility#1 [ff6000001fc50320] panic at ffffffff808773ec
 crash-utility#2 [ff6000001fc50380] walk_stackframe at ffffffff800056da
     PC: ffffffff80876a34  [memset+96]
     RA: ffffffff80563dc0  [recursive_loop+68]
     SP: ff2000000000fd50  CAUSE: 000000000000000f
epc : ffffffff80876a34 ra : ffffffff80563dc0 sp : ff2000000000fd50
 gp : ffffffff81515d38 tp : 0000000000000000 t0 : ff2000000000fd58
 t1 : ff600000000d88c8 t2 : 6143203a6d74646b s0 : ff20000000010190
 s1 : 0000000000000012 a0 : ff2000000000fd58 a1 : 1212121212121212
 a2 : 0000000000000400 a3 : ff20000000010158 a4 : 0000000000000000
 a5 : 725bedba92260900 a6 : 000000000130e0f0 a7 : 0000000000000000
 s2 : ff2000000000fd58 s3 : ffffffff815170d8 s4 : ff20000000013e60
 s5 : 000000000000000e s6 : ff20000000013e60 s7 : 0000000000000000
 s8 : ff60000000861000 s9 : 00007fffc3641694 s10: 00007fffc3641690
 s11: 00005555796ed240 t3 : 0000000000010297 t4 : ffffffff80c17810
 t5 : ffffffff8195e7b8 t6 : ff20000000013b18
 status: 0000000200000120 badaddr: ff2000000000fd58
  cause: 000000000000000f orig_a0: 0000000000000000
--- <OVERFLOW stack> ---
 crash-utility#3 [ff2000000000fd50] memset at ffffffff80876a34
 crash-utility#4 [ff20000000010190] recursive_loop at ffffffff80563e16
 crash-utility#5 [ff200000000105d0] recursive_loop at ffffffff80563e16
 < recursive_loop ...>
 crash-utility#16 [ff20000000013490] recursive_loop at ffffffff80563e16
 crash-utility#17 [ff200000000138d0] recursive_loop at ffffffff80563e16
 crash-utility#18 [ff20000000013d10] lkdtm_EXHAUST_STACK at ffffffff8088005e
 crash-utility#19 [ff20000000013d30] lkdtm_do_action at ffffffff80563292
 crash-utility#20 [ff20000000013d40] direct_entry at ffffffff80563474
 crash-utility#21 [ff20000000013d70] full_proxy_write at ffffffff8032fb3a
 crash-utility#22 [ff20000000013db0] vfs_write at ffffffff801d6414
 crash-utility#23 [ff20000000013e60] ksys_write at ffffffff801d67b8
 crash-utility#24 [ff20000000013eb0] __riscv_sys_write at ffffffff801d6832
 crash-utility#25 [ff20000000013ec0] do_trap_ecall_u at ffffffff80884a20
crash>

crash> help -m
<snip>
        irq_stack_size: 16384
         irq_stacks[0]: ff20000000000000
         irq_stacks[1]: ff20000000008000
        overflow_stack_size: 4096
         overflow_stacks[0]: ff6000001fa7a510
         overflow_stacks[1]: ff6000001fc4f510
crash>
```
Signed-off-by: Song Shuai <[email protected]>
k-hagio pushed a commit that referenced this issue Jan 11, 2024
The patch introduces per-cpu overflow stacks for RISCV64 to let
"bt" do backtrace on it and the 'help -m' command dispalys the
addresss of each per-cpu overflow stack.

TEST: a lkdtm DIRECT EXHAUST_STACK vmcore

  crash> bt
  PID: 1        TASK: ff600000000d8000  CPU: 1    COMMAND: "sh"
   #0 [ff6000001fc501c0] riscv_crash_save_regs at ffffffff8000a1dc
   #1 [ff6000001fc50320] panic at ffffffff808773ec
   #2 [ff6000001fc50380] walk_stackframe at ffffffff800056da
       PC: ffffffff80876a34  [memset+96]
       RA: ffffffff80563dc0  [recursive_loop+68]
       SP: ff2000000000fd50  CAUSE: 000000000000000f
  epc : ffffffff80876a34 ra : ffffffff80563dc0 sp : ff2000000000fd50
   gp : ffffffff81515d38 tp : 0000000000000000 t0 : ff2000000000fd58
   t1 : ff600000000d88c8 t2 : 6143203a6d74646b s0 : ff20000000010190
   s1 : 0000000000000012 a0 : ff2000000000fd58 a1 : 1212121212121212
   a2 : 0000000000000400 a3 : ff20000000010158 a4 : 0000000000000000
   a5 : 725bedba92260900 a6 : 000000000130e0f0 a7 : 0000000000000000
   s2 : ff2000000000fd58 s3 : ffffffff815170d8 s4 : ff20000000013e60
   s5 : 000000000000000e s6 : ff20000000013e60 s7 : 0000000000000000
   s8 : ff60000000861000 s9 : 00007fffc3641694 s10: 00007fffc3641690
   s11: 00005555796ed240 t3 : 0000000000010297 t4 : ffffffff80c17810
   t5 : ffffffff8195e7b8 t6 : ff20000000013b18
   status: 0000000200000120 badaddr: ff2000000000fd58
    cause: 000000000000000f orig_a0: 0000000000000000
  --- <OVERFLOW stack> ---
   #3 [ff2000000000fd50] memset at ffffffff80876a34
   #4 [ff20000000010190] recursive_loop at ffffffff80563e16
   #5 [ff200000000105d0] recursive_loop at ffffffff80563e16
   < recursive_loop ...>
   #16 [ff20000000013490] recursive_loop at ffffffff80563e16
   #17 [ff200000000138d0] recursive_loop at ffffffff80563e16
   #18 [ff20000000013d10] lkdtm_EXHAUST_STACK at ffffffff8088005e
   #19 [ff20000000013d30] lkdtm_do_action at ffffffff80563292
   #20 [ff20000000013d40] direct_entry at ffffffff80563474
   #21 [ff20000000013d70] full_proxy_write at ffffffff8032fb3a
   #22 [ff20000000013db0] vfs_write at ffffffff801d6414
   #23 [ff20000000013e60] ksys_write at ffffffff801d67b8
   #24 [ff20000000013eb0] __riscv_sys_write at ffffffff801d6832
   #25 [ff20000000013ec0] do_trap_ecall_u at ffffffff80884a20
  crash>

  crash> help -m
  <snip>
          irq_stack_size: 16384
           irq_stacks[0]: ff20000000000000
           irq_stacks[1]: ff20000000008000
          overflow_stack_size: 4096
           overflow_stacks[0]: ff6000001fa7a510
           overflow_stacks[1]: ff6000001fc4f510
  crash>

Signed-off-by: Song Shuai <[email protected]>
liutgnu pushed a commit to liutgnu/crash-preview that referenced this issue Feb 21, 2024
The patch introduces per-cpu overflow stacks for RISCV64 to let
"bt" do backtrace on it and the 'help -m' command dispalys the
addresss of each per-cpu overflow stack.

TEST: a lkdtm DIRECT EXHAUST_STACK vmcore

  crash> bt
  PID: 1        TASK: ff600000000d8000  CPU: 1    COMMAND: "sh"
   #0 [ff6000001fc501c0] riscv_crash_save_regs at ffffffff8000a1dc
   crash-utility#1 [ff6000001fc50320] panic at ffffffff808773ec
   crash-utility#2 [ff6000001fc50380] walk_stackframe at ffffffff800056da
       PC: ffffffff80876a34  [memset+96]
       RA: ffffffff80563dc0  [recursive_loop+68]
       SP: ff2000000000fd50  CAUSE: 000000000000000f
  epc : ffffffff80876a34 ra : ffffffff80563dc0 sp : ff2000000000fd50
   gp : ffffffff81515d38 tp : 0000000000000000 t0 : ff2000000000fd58
   t1 : ff600000000d88c8 t2 : 6143203a6d74646b s0 : ff20000000010190
   s1 : 0000000000000012 a0 : ff2000000000fd58 a1 : 1212121212121212
   a2 : 0000000000000400 a3 : ff20000000010158 a4 : 0000000000000000
   a5 : 725bedba92260900 a6 : 000000000130e0f0 a7 : 0000000000000000
   s2 : ff2000000000fd58 s3 : ffffffff815170d8 s4 : ff20000000013e60
   s5 : 000000000000000e s6 : ff20000000013e60 s7 : 0000000000000000
   s8 : ff60000000861000 s9 : 00007fffc3641694 s10: 00007fffc3641690
   s11: 00005555796ed240 t3 : 0000000000010297 t4 : ffffffff80c17810
   t5 : ffffffff8195e7b8 t6 : ff20000000013b18
   status: 0000000200000120 badaddr: ff2000000000fd58
    cause: 000000000000000f orig_a0: 0000000000000000
  --- <OVERFLOW stack> ---
   crash-utility#3 [ff2000000000fd50] memset at ffffffff80876a34
   crash-utility#4 [ff20000000010190] recursive_loop at ffffffff80563e16
   crash-utility#5 [ff200000000105d0] recursive_loop at ffffffff80563e16
   < recursive_loop ...>
   crash-utility#16 [ff20000000013490] recursive_loop at ffffffff80563e16
   crash-utility#17 [ff200000000138d0] recursive_loop at ffffffff80563e16
   crash-utility#18 [ff20000000013d10] lkdtm_EXHAUST_STACK at ffffffff8088005e
   crash-utility#19 [ff20000000013d30] lkdtm_do_action at ffffffff80563292
   crash-utility#20 [ff20000000013d40] direct_entry at ffffffff80563474
   crash-utility#21 [ff20000000013d70] full_proxy_write at ffffffff8032fb3a
   crash-utility#22 [ff20000000013db0] vfs_write at ffffffff801d6414
   crash-utility#23 [ff20000000013e60] ksys_write at ffffffff801d67b8
   crash-utility#24 [ff20000000013eb0] __riscv_sys_write at ffffffff801d6832
   crash-utility#25 [ff20000000013ec0] do_trap_ecall_u at ffffffff80884a20
  crash>

  crash> help -m
  <snip>
          irq_stack_size: 16384
           irq_stacks[0]: ff20000000000000
           irq_stacks[1]: ff20000000008000
          overflow_stack_size: 4096
           overflow_stacks[0]: ff6000001fa7a510
           overflow_stacks[1]: ff6000001fc4f510
  crash>

Signed-off-by: Song Shuai <[email protected]>
k-hagio pushed a commit that referenced this issue Mar 28, 2024
…ame" warning

The "bogus exception frame" warning was observed again on a specific
vmcore, and the remaining frame was truncated on x86_64 machine, when
executing the "bt" command as below:

  crash> bt 0 -c 8
  PID: 0        TASK: ffff9948c08f5640  CPU: 8    COMMAND: "swapper/8"
   #0 [fffffe1788788e58] crash_nmi_callback at ffffffff972672bb
   #1 [fffffe1788788e68] nmi_handle at ffffffff9722eb8e
   #2 [fffffe1788788eb0] default_do_nmi at ffffffff97e51cd0
   #3 [fffffe1788788ed0] exc_nmi at ffffffff97e51ee1
   #4 [fffffe1788788ef0] end_repeat_nmi at ffffffff980015f9
      [exception RIP: __update_load_avg_se+13]
      RIP: ffffffff9736b16d  RSP: ffffbec3c08acc78  RFLAGS: 00000046
      RAX: 0000000000000000  RBX: ffff994c2f2b1a40  RCX: ffffbec3c08acdc0
      RDX: ffff9948e4fe1d80  RSI: ffff994c2f2b1a40  RDI: 0000001d7ad7d55d
      RBP: ffffbec3c08acc88   R8: 0000001d921fca6f   R9: ffff994c2f2b1328
      R10: 00000000fffd0010  R11: ffffffff98e060c0  R12: 0000001d7ad7d55d
      R13: 0000000000000005  R14: ffff994c2f2b19c0  R15: 0000000000000001
      ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
  --- <NMI exception stack> ---
   #5 [ffffbec3c08acc78] __update_load_avg_se at ffffffff9736b16d
   #6 [ffffbec3c08acce0] enqueue_entity at ffffffff9735c9ab
   #7 [ffffbec3c08acd28] enqueue_task_fair at ffffffff9735cef8
  ...
  #18 [ffffbec3c08acf90] blk_complete_reqs at ffffffff977978d0
  #19 [ffffbec3c08acfa0] __do_softirq at ffffffff97e66f7a
  #20 [ffffbec3c08acff0] do_softirq at ffffffff9730f6ef
  --- <IRQ stack> ---
  #21 [ffffbec3c022ff18] do_idle at ffffffff97368288
      [exception RIP: unknown or invalid address]
      RIP: 0000000000000000  RSP: 0000000000000000  RFLAGS: 00000000
      RAX: 0000000000000000  RBX: 000000089726a2d0  RCX: 0000000000000000
      RDX: 0000000000000000  RSI: 0000000000000000  RDI: 0000000000000000
      RBP: ffffffff9726a3dd   R8: 0000000000000000   R9: 0000000000000000
      R10: ffffffff9720015a  R11: e48885e126bc1600  R12: 0000000000000000
      R13: ffffffff973684a9  R14: 0000000000000094  R15: 0000000040000000
      ORIG_RAX: 0000000000000000  CS: 0000  SS: 0000
  bt: WARNING: possibly bogus exception frame
  crash>

Actually there is no exception frame, when called from do_softirq().
With the patch:

  crash> bt 0 -c 8
  ...
  #18 [ffffbec3c08acf90] blk_complete_reqs at ffffffff977978d0
  #19 [ffffbec3c08acfa0] __do_softirq at ffffffff97e66f7a
  #20 [ffffbec3c08acff0] do_softirq at ffffffff9730f6ef
  --- <IRQ stack> ---
  #21 [ffffbec3c022ff28] cpu_startup_entry at ffffffff973684a9
  #22 [ffffbec3c022ff38] start_secondary at ffffffff9726a3dd
  #23 [ffffbec3c022ff50] secondary_startup_64_no_verify at ffffffff9720015a
  crash>

Reported-by: Jie Li <[email protected]>
Signed-off-by: Lianbo Jiang <[email protected]>
liutgnu added a commit to liutgnu/crash-preview that referenced this issue Dec 5, 2024
…usly

There is an issue that, for kernel modules, "dis -rl" fails to display
modules code line number data after execute "bt" command in crash.

Without the patch:
  crsah> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp
  0xffffffffc0f60eb6 <lpfc_nlp_get+6>:    push   %rbx
  0xffffffffc0f60eb7 <lpfc_nlp_get+7>:    test   %rdi,%rdi

With the patch:
  crash> mod -S
  crash> bt
  PID: 1500     TASK: ff2bd8b093524000  CPU: 16   COMMAND: "lpfc_worker_0"
   #0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
   ...snip...
   crash-utility#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
   ...snip...
  crash> dis -rl ffffffffc0f60f82
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6756
  0xffffffffc0f60eb0 <lpfc_nlp_get>:      nopl   0x0(%rax,%rax,1) [FTRACE NOP]
  /usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6759
  0xffffffffc0f60eb5 <lpfc_nlp_get+5>:    push   %rbp

The root cause is, after kernel module been loaded by mod command, the symtable
is not expanded in gdb side. crash bt or dis command will trigger such an
expansion. However the symtable expansion is different for the 2 commands:

The stack trace of "dis -rl" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f75e in dw2_expand_all_symtabs ...
  crash-utility#8  0x00000000008f254d in gdb_get_line_number ...
  crash-utility#9  0x00000000008f22af in gdb_command_funnel_1 ...
  crash-utility#10 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#11 0x00000000005b7f02 in gdb_interface ...
  crash-utility#12 0x00000000005f5bd8 in get_line_number ...
  crash-utility#13 0x000000000059e574 in cmd_dis ...

The stack trace of "bt" for symtable expanding:

  #0  0x00000000008d8d9f in add_compunit_symtab_to_objfile ...
  crash-utility#1  0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector ...
  crash-utility#2  0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block ...
  crash-utility#3  0x000000000077e8e9 in process_full_comp_unit ...
  crash-utility#4  process_queue ...
  crash-utility#5  dw2_do_instantiate_symtab ...
  crash-utility#6  0x000000000077ed67 in dw2_instantiate_symtab ...
  crash-utility#7  0x000000000077f8ed in dw2_lookup_symbol ...
  crash-utility#8  0x00000000008e6d03 in lookup_symbol_via_quick_fns ...
  crash-utility#9  0x00000000008e7153 in lookup_symbol_in_objfile ...
  crash-utility#10 0x00000000008e73c6 in lookup_symbol_global_or_static_iterator_cb ...
  crash-utility#11 0x00000000008b99c4 in svr4_iterate_over_objfiles_in_search_order ...
  crash-utility#12 0x00000000008e754e in lookup_global_or_static_symbol ...
  crash-utility#13 0x00000000008e75da in lookup_static_symbol ...
  crash-utility#14 0x00000000008e632c in lookup_symbol_aux ...
  crash-utility#15 0x00000000008e5a7a in lookup_symbol_in_language ...
  crash-utility#16 0x00000000008e5b30 in lookup_symbol ...
  crash-utility#17 0x00000000008f2a4a in gdb_get_datatype ...
  crash-utility#18 0x00000000008f22c0 in gdb_command_funnel_1 ...
  crash-utility#19 0x00000000008f2003 in gdb_command_funnel ...
  crash-utility#20 0x00000000005b7f02 in gdb_interface ...
  crash-utility#21 0x00000000005f8a9f in datatype_info ...
  crash-utility#22 0x0000000000599947 in cpu_map_size ...
  crash-utility#23 0x00000000005a975d in get_cpus_online ...
  crash-utility#24 0x0000000000637a8b in diskdump_get_prstatus_percpu ...
  crash-utility#25 0x000000000062f0e4 in get_netdump_regs_x86_64 ...
  crash-utility#26 0x000000000059fe68 in back_trace ...
  crash-utility#27 0x00000000005ab1cb in cmd_bt ...

For the stacktrace of "dis -rl", it calls dw2_expand_all_symtabs() to expand
all symtable of the objfile, or "*.ko.debug" in our case. However for
the stacktrace of "bt", it doesn't expand all, but only a subset of symtable
which is enough to find a symbol by dw2_lookup_symbol(). As a result, the
objfile->compunit_symtabs, which is the head of a single linked list of
struct compunit_symtab, is not NULL but didn't contain all symtables. It
will not be reinitialized in gdb_get_line_number() by "dis -rl" because
!objfile_has_full_symbols(objfile) check will fail, so it cannot display
the proper code line number data.

Since objfile_has_full_symbols(objfile) check cannot ensure all symbols
been expanded, this patch add a new member as a flag for struct objfile
to record if all symbols have been expanded. The flag will be set only ofter
expand_all_symtabs been called.

Signed-off-by: Tao Liu <[email protected]>
liutgnu pushed a commit to liutgnu/crash-preview that referenced this issue Dec 5, 2024
The patch introduces per-cpu overflow stacks for RISCV64 to let
"bt" do backtrace on it and the 'help -m' command dispalys the
addresss of each per-cpu overflow stack.

TEST: a lkdtm DIRECT EXHAUST_STACK vmcore

  crash> bt
  PID: 1        TASK: ff600000000d8000  CPU: 1    COMMAND: "sh"
   #0 [ff6000001fc501c0] riscv_crash_save_regs at ffffffff8000a1dc
   crash-utility#1 [ff6000001fc50320] panic at ffffffff808773ec
   crash-utility#2 [ff6000001fc50380] walk_stackframe at ffffffff800056da
       PC: ffffffff80876a34  [memset+96]
       RA: ffffffff80563dc0  [recursive_loop+68]
       SP: ff2000000000fd50  CAUSE: 000000000000000f
  epc : ffffffff80876a34 ra : ffffffff80563dc0 sp : ff2000000000fd50
   gp : ffffffff81515d38 tp : 0000000000000000 t0 : ff2000000000fd58
   t1 : ff600000000d88c8 t2 : 6143203a6d74646b s0 : ff20000000010190
   s1 : 0000000000000012 a0 : ff2000000000fd58 a1 : 1212121212121212
   a2 : 0000000000000400 a3 : ff20000000010158 a4 : 0000000000000000
   a5 : 725bedba92260900 a6 : 000000000130e0f0 a7 : 0000000000000000
   s2 : ff2000000000fd58 s3 : ffffffff815170d8 s4 : ff20000000013e60
   s5 : 000000000000000e s6 : ff20000000013e60 s7 : 0000000000000000
   s8 : ff60000000861000 s9 : 00007fffc3641694 s10: 00007fffc3641690
   s11: 00005555796ed240 t3 : 0000000000010297 t4 : ffffffff80c17810
   t5 : ffffffff8195e7b8 t6 : ff20000000013b18
   status: 0000000200000120 badaddr: ff2000000000fd58
    cause: 000000000000000f orig_a0: 0000000000000000
  --- <OVERFLOW stack> ---
   crash-utility#3 [ff2000000000fd50] memset at ffffffff80876a34
   crash-utility#4 [ff20000000010190] recursive_loop at ffffffff80563e16
   crash-utility#5 [ff200000000105d0] recursive_loop at ffffffff80563e16
   < recursive_loop ...>
   crash-utility#16 [ff20000000013490] recursive_loop at ffffffff80563e16
   crash-utility#17 [ff200000000138d0] recursive_loop at ffffffff80563e16
   crash-utility#18 [ff20000000013d10] lkdtm_EXHAUST_STACK at ffffffff8088005e
   crash-utility#19 [ff20000000013d30] lkdtm_do_action at ffffffff80563292
   crash-utility#20 [ff20000000013d40] direct_entry at ffffffff80563474
   crash-utility#21 [ff20000000013d70] full_proxy_write at ffffffff8032fb3a
   crash-utility#22 [ff20000000013db0] vfs_write at ffffffff801d6414
   crash-utility#23 [ff20000000013e60] ksys_write at ffffffff801d67b8
   crash-utility#24 [ff20000000013eb0] __riscv_sys_write at ffffffff801d6832
   crash-utility#25 [ff20000000013ec0] do_trap_ecall_u at ffffffff80884a20
  crash>

  crash> help -m
  <snip>
          irq_stack_size: 16384
           irq_stacks[0]: ff20000000000000
           irq_stacks[1]: ff20000000008000
          overflow_stack_size: 4096
           overflow_stacks[0]: ff6000001fa7a510
           overflow_stacks[1]: ff6000001fc4f510
  crash>

Signed-off-by: Song Shuai <[email protected]>
liutgnu pushed a commit to liutgnu/crash-preview that referenced this issue Dec 5, 2024
…ame" warning

The "bogus exception frame" warning was observed again on a specific
vmcore, and the remaining frame was truncated on x86_64 machine, when
executing the "bt" command as below:

  crash> bt 0 -c 8
  PID: 0        TASK: ffff9948c08f5640  CPU: 8    COMMAND: "swapper/8"
   #0 [fffffe1788788e58] crash_nmi_callback at ffffffff972672bb
   crash-utility#1 [fffffe1788788e68] nmi_handle at ffffffff9722eb8e
   crash-utility#2 [fffffe1788788eb0] default_do_nmi at ffffffff97e51cd0
   crash-utility#3 [fffffe1788788ed0] exc_nmi at ffffffff97e51ee1
   crash-utility#4 [fffffe1788788ef0] end_repeat_nmi at ffffffff980015f9
      [exception RIP: __update_load_avg_se+13]
      RIP: ffffffff9736b16d  RSP: ffffbec3c08acc78  RFLAGS: 00000046
      RAX: 0000000000000000  RBX: ffff994c2f2b1a40  RCX: ffffbec3c08acdc0
      RDX: ffff9948e4fe1d80  RSI: ffff994c2f2b1a40  RDI: 0000001d7ad7d55d
      RBP: ffffbec3c08acc88   R8: 0000001d921fca6f   R9: ffff994c2f2b1328
      R10: 00000000fffd0010  R11: ffffffff98e060c0  R12: 0000001d7ad7d55d
      R13: 0000000000000005  R14: ffff994c2f2b19c0  R15: 0000000000000001
      ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
  --- <NMI exception stack> ---
   crash-utility#5 [ffffbec3c08acc78] __update_load_avg_se at ffffffff9736b16d
   crash-utility#6 [ffffbec3c08acce0] enqueue_entity at ffffffff9735c9ab
   crash-utility#7 [ffffbec3c08acd28] enqueue_task_fair at ffffffff9735cef8
  ...
  crash-utility#18 [ffffbec3c08acf90] blk_complete_reqs at ffffffff977978d0
  crash-utility#19 [ffffbec3c08acfa0] __do_softirq at ffffffff97e66f7a
  crash-utility#20 [ffffbec3c08acff0] do_softirq at ffffffff9730f6ef
  --- <IRQ stack> ---
  crash-utility#21 [ffffbec3c022ff18] do_idle at ffffffff97368288
      [exception RIP: unknown or invalid address]
      RIP: 0000000000000000  RSP: 0000000000000000  RFLAGS: 00000000
      RAX: 0000000000000000  RBX: 000000089726a2d0  RCX: 0000000000000000
      RDX: 0000000000000000  RSI: 0000000000000000  RDI: 0000000000000000
      RBP: ffffffff9726a3dd   R8: 0000000000000000   R9: 0000000000000000
      R10: ffffffff9720015a  R11: e48885e126bc1600  R12: 0000000000000000
      R13: ffffffff973684a9  R14: 0000000000000094  R15: 0000000040000000
      ORIG_RAX: 0000000000000000  CS: 0000  SS: 0000
  bt: WARNING: possibly bogus exception frame
  crash>

Actually there is no exception frame, when called from do_softirq().
With the patch:

  crash> bt 0 -c 8
  ...
  crash-utility#18 [ffffbec3c08acf90] blk_complete_reqs at ffffffff977978d0
  crash-utility#19 [ffffbec3c08acfa0] __do_softirq at ffffffff97e66f7a
  crash-utility#20 [ffffbec3c08acff0] do_softirq at ffffffff9730f6ef
  --- <IRQ stack> ---
  crash-utility#21 [ffffbec3c022ff28] cpu_startup_entry at ffffffff973684a9
  crash-utility#22 [ffffbec3c022ff38] start_secondary at ffffffff9726a3dd
  crash-utility#23 [ffffbec3c022ff50] secondary_startup_64_no_verify at ffffffff9720015a
  crash>

Reported-by: Jie Li <[email protected]>
Signed-off-by: Lianbo Jiang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants