Update the section on debugging a kernel with GDB to include a recent

dump and the basics of what most developers would like to see in reports
of a panic where a dump is available.  Also trim a few lines from the
previous dump example (still included) and sync up some of the gdb
commands for the sake of consistency.

Inspired by:	Recent exchange with jhb
This commit is contained in:
Sean Chittenden 2003-12-04 01:46:33 +00:00
parent 9030b241d7
commit 10be000a0a
Notes: svn2git 2020-12-08 03:00:23 +00:00
svn path=/head/; revision=19043

View file

@ -143,19 +143,127 @@
<filename>/usr/obj/usr/src/sys/KERNCONF</filename>, where
<filename>KERNCONF</filename> is the <varname>ident</varname>
specified in a kernel &man.config.5;). With those two pieces of
info, let the debugging commence!
info, let the debugging commence!</para>
<para>To enter into the debugger and begin getting information
from the dump, the following steps are required at a minimum:
<screen>&prompt.root; <userinput>cd /usr/obj/usr/src/sys/KERNCONF</userinput>
&prompt.root; <userinput>gdb -k /boot/kernel/kernel.debug /var/crash/vmcore.0</userinput></screen>
and voila! You can debug the crash dump using the kernel
sources just like you can for any other program.</para>
You can debug the crash dump using the kernel sources just like
you can for any other program.</para>
<para>Here is a script log of a <command>gdb</command> session
illustrating the procedure. Long lines have been folded to improve
readability, and the lines are numbered for reference. Despite this, it
is a real-world error trace taken during the development of the pcvt
console driver.</para>
<para>This first dump is from a 5.2-BETA kernel and the crash
comes from deep within the kernel. The ouput below has been
modified to include line numbers on the left. This first trace
inspects the instruction pointer and obtains a back trace. The
address that is used on line 41 for the <command>list</command>
command is the instruction pointer and can be found on line line
17. Most developers will request having at least this
information sent to them if you are unable to debug the problem
yourself. If, however, you do solve the problem, make sure that
your patch winds its way into the source tree via a problem
report, mailing lists, or by being able to commit it!
<screen> 1:&prompt.root; <userinput>cd /usr/obj/usr/src/sys/KERNCONF</userinput>
2:&prompt.root; <userinput>gdb -k kernel.debug /var/crash/vmcore.0</userinput>
3:GNU gdb 5.2.1 (FreeBSD)
4:Copyright 2002 Free Software Foundation, Inc.
5:GDB is free software, covered by the GNU General Public License, and you are
6:welcome to change it and/or distribute copies of it under certain conditions.
7:Type "show copying" to see the conditions.
8:There is absolutely no warranty for GDB. Type "show warranty" for details.
9:This GDB was configured as "i386-undermydesk-freebsd"...
10:panic: page fault
11:panic messages:
12:---
13:Fatal trap 12: page fault while in kernel mode
14:cpuid = 0; apic id = 00
15:fault virtual address = 0x300
16:fault code: = supervisor read, page not present
17:instruction pointer = 0x8:0xc0713860
18:stack pointer = 0x10:0xdc1d0b70
19:frame pointer = 0x10:0xdc1d0b7c
20:code segment = base 0x0, limit 0xfffff, type 0x1b
21: = DPL 0, pres 1, def32 1, gran 1
22:processor eflags = resume, IOPL = 0
23:current process = 14394 (uname)
24:trap number = 12
25:panic: page fault
26 cpuid = 0;
27:Stack backtrace:
28
29:syncing disks, buffers remaining... 2199 2199 panic: mi_switch: switch in a critical section
30:cpuid = 0;
31:Uptime: 2h43m19s
32:Dumping 255 MB
33: 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
34:---
35:Reading symbols from /boot/kernel/snd_maestro3.ko...done.
36:Loaded symbols for /boot/kernel/snd_maestro3.ko
37:Reading symbols from /boot/kernel/snd_pcm.ko...done.
38:Loaded symbols for /boot/kernel/snd_pcm.ko
39:#0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240
40:240 dumping++;
41:<prompt>(kgdb)</prompt> <userinput>list *0xc0713860</userinput>
42:0xc0713860 is in lapic_ipi_wait (/usr/src/sys/i386/i386/local_apic.c:663).
43:658 incr = 0;
44:659 delay = 1;
45:660 } else
46:661 incr = 1;
47:662 for (x = 0; x &lt; delay; x += incr) {
48:663 if ((lapic-&gt;icr_lo &amp; APIC_DELSTAT_MASK) == APIC_DELSTAT_IDLE)
49:664 return (1);
50:665 ia32_pause();
51:666 }
52:667 return (0);
53:<prompt>(kgdb)</prompt> <userinput>backtrace</userinput>
54:#0 doadump () at /usr/src/sys/kern/kern_shutdown.c:240
55:#1 0xc055fd9b in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:372
56:#2 0xc056019d in panic () at /usr/src/sys/kern/kern_shutdown.c:550
57:#3 0xc0567ef5 in mi_switch () at /usr/src/sys/kern/kern_synch.c:470
58:#4 0xc055fa87 in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:312
59:#5 0xc056019d in panic () at /usr/src/sys/kern/kern_shutdown.c:550
60:#6 0xc0720c66 in trap_fatal (frame=0xdc1d0b30, eva=0)
61: at /usr/src/sys/i386/i386/trap.c:821
62:#7 0xc07202b3 in trap (frame=
63: {tf_fs = -1065484264, tf_es = -1065484272, tf_ds = -1065484272, tf_edi = 1, tf_esi = 0, tf_ebp = -602076292, tf_isp = -602076324, tf_ebx = 0, tf_edx = 0, tf_ecx = 1000000, tf_eax = 243, tf_trapno = 12, tf_err = 0, tf_eip = -1066321824, tf_cs = 8, tf_eflags = 65671, tf_esp = 243, tf_ss = 0})
64: at /usr/src/sys/i386/i386/trap.c:250
65:#8 0xc070c9f8 in calltrap () at {standard input}:94
66:#9 0xc07139f3 in lapic_ipi_vectored (vector=0, dest=0)
67: at /usr/src/sys/i386/i386/local_apic.c:733
68:#10 0xc0718b23 in ipi_selected (cpus=1, ipi=1)
69: at /usr/src/sys/i386/i386/mp_machdep.c:1115
70:#11 0xc057473e in kseq_notify (ke=0xcc05e360, cpu=0)
71: at /usr/src/sys/kern/sched_ule.c:520
72:#12 0xc0575cad in sched_add (td=0xcbcf5c80)
73: at /usr/src/sys/kern/sched_ule.c:1366
74:#13 0xc05666c6 in setrunqueue (td=0xcc05e360)
75: at /usr/src/sys/kern/kern_switch.c:422
76:#14 0xc05752f4 in sched_wakeup (td=0xcbcf5c80)
77: at /usr/src/sys/kern/sched_ule.c:999
78:#15 0xc056816c in setrunnable (td=0xcbcf5c80)
79: at /usr/src/sys/kern/kern_synch.c:570
80:#16 0xc0567d53 in wakeup (ident=0xcbcf5c80)
81: at /usr/src/sys/kern/kern_synch.c:411
82:#17 0xc05490a8 in exit1 (td=0xcbcf5b40, rv=0)
83: at /usr/src/sys/kern/kern_exit.c:509
84:#18 0xc0548011 in sys_exit () at /usr/src/sys/kern/kern_exit.c:102
85:#19 0xc0720fd0 in syscall (frame=
86: {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = -1, tf_ebp = -1077940712, tf_isp = -602075788, tf_ebx = 672411944, tf_edx = 10, tf_ecx = 672411600, tf_eax = 1, tf_trapno = 12, tf_err = 2, tf_eip = 671899563, tf_cs = 31, tf_eflags = 642, tf_esp = -1077940740, tf_ss = 47})
87: at /usr/src/sys/i386/i386/trap.c:1010
88:#20 0xc070ca4d in Xint0x80_syscall () at {standard input}:136
89:---Can't read userspace from dump, or kernel process---
90:<prompt>(kgdb)</prompt> <userinput>quit</userinput></screen></para>
<para>This next trace is an older dump from the FreeBSD 2 time
frame, but is more involved and demonstrates more of the
features of <command>gdb</command>. Long lines have been folded
to improve readability, and the lines are numbered for
reference. Despite this, it is a real-world error trace taken
during the development of the pcvt console driver.</para>
<screen> 1:Script started on Fri Dec 30 23:15:22 1994
2:&prompt.root; <userinput>cd /sys/compile/URIAH</userinput>
@ -166,7 +274,7 @@
6:panic: because you said to!
7:current pcb at 1e3f70
8:Reading in symbols for ../../i386/i386/machdep.c...done.
9:<prompt>(kgdb)</prompt> <userinput>where</userinput>
9:<prompt>(kgdb)</prompt> <userinput>backtrace</userinput>
10:#0 boot (arghowto=256) (../../i386/i386/machdep.c line 767)
11:#1 0xf0115159 in panic ()
12:#2 0xf01955bd in diediedie () (../../i386/i386/machdep.c line 698)
@ -233,11 +341,7 @@
73:673 error = (*callp-&gt;sy_call)(p, args, rval);
74:<prompt>(kgdb)</prompt> <userinput>up</userinput>
75:Initial frame selected; you cannot go up.
76:<prompt>(kgdb)</prompt> <userinput>quit</userinput>
77:&prompt.root; <userinput>exit</userinput>
78:exit
79:
80:Script done on Fri Dec 30 23:18:04 1994</screen>
76:<prompt>(kgdb)</prompt> <userinput>quit</userinput></screen>
<para>Comments to the above script:</para>
<variablelist>