1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
<HEAD>
<META NAME="GENERATOR" CONTENT="LinuxDoc-Tools 0.9.18">
<TITLE>User Mode Linux HOWTO : Kernel debugging</TITLE>
<LINK HREF="UserModeLinux-HOWTO-10.html" REL=next>
<LINK HREF="UserModeLinux-HOWTO-8.html" REL=previous>
<LINK HREF="UserModeLinux-HOWTO.html#toc9" REL=contents>
</HEAD>
<BODY>
<A HREF="UserModeLinux-HOWTO-10.html">Next</A>
<A HREF="UserModeLinux-HOWTO-8.html">Previous</A>
<A HREF="UserModeLinux-HOWTO.html#toc9">Contents</A>
<HR>
<H2><A NAME="debugging"></A> <A NAME="s9">9.</A> <A HREF="UserModeLinux-HOWTO.html#toc9">Kernel debugging</A></H2>
<P>
<B>Note:</B> The interface that makes debugging, as described here,
possible is present in 2.4.0-test6 kernels and later.</P>
<P>
Since the user-mode kernel runs as a normal Linux process, it is
possible to debug it with gdb almost like any other process. It is
slightly different because the kernel's threads are already being
ptraced for system call interception, so gdb can't ptrace them.
However, a mechanism has been added to work around that problem.</P>
<P>
In order to debug the kernel, you need build it from source. See
<A HREF="UserModeLinux-HOWTO-2.html#compile">Compiling the kernel and modules</A>
for information on doing that.</P>
<P> </P>
<H2><A NAME="ss9.1">9.1</A> <A HREF="UserModeLinux-HOWTO.html#toc9.1">Starting the kernel under gdb</A>
</H2>
<P>You can have the kernel running under the control of gdb from the
beginning by putting 'debug' on the command line. You will get an
xterm with gdb running inside it. The kernel will send some commands
to gdb which will leave it stopped at the beginning of start_kernel.
At this point, you can get things going with 'next', 'step', or 'cont'.</P>
<P>
There is a transcript of a debugging session
<A HREF="debug-session.html">here</A> , with breakpoints being set in the
scheduler and in an interrupt handler.</P>
<H2><A NAME="ss9.2">9.2</A> <A HREF="UserModeLinux-HOWTO.html#toc9.2">Attaching gdb to the kernel</A>
</H2>
<P>If you don't have the kernel running under gdb, you can attach gdb to
it later by sending the tracing thread a SIGUSR1. The first line of
the console output identifies its pid:
<BLOCKQUOTE><CODE>
<PRE>
tracing thread pid = 20093
</PRE>
</CODE></BLOCKQUOTE>
When you send it the signal:
<BLOCKQUOTE><CODE>
<PRE>
kill -USR1 20093
</PRE>
</CODE></BLOCKQUOTE>
you will get an xterm with gdb running in it.</P>
<P>
If you have the mconsole compiled into UML, then the mconsole client
can be used to start gdb:
<BLOCKQUOTE><CODE>
<PRE>
(mconsole) config gdb=xterm
</PRE>
</CODE></BLOCKQUOTE>
will fire up an xterm with gdb running in it.</P>
<H2><A NAME="ss9.3">9.3</A> <A HREF="UserModeLinux-HOWTO.html#toc9.3">Debugging modules</A>
</H2>
<P>gdb has support for debugging code which is dynamically loaded into
the process. This support is what is needed to debug kernel modules
under UML.</P>
<P>Boot the kernel under the debugger and load the module with insmod or
modprobe. With gdb, do:
<BLOCKQUOTE><CODE>
<PRE>
(gdb) p module_list
</PRE>
</CODE></BLOCKQUOTE>
This is a list of modules that have been loaded into the kernel, with
the most recently loaded module first. Normally, the module you want
is at module_list. If it's not, walk down the next links, looking at
the name fields until find the module you want to debug. Take the
address of that structure, and add module.size_of_struct (which in
2.4.10 kernels is 96 (0x60)) to it. Gdb can make this hard addition for
you :-):
<BLOCKQUOTE><CODE>
<PRE>
printf "%#x\n", (int)module_list module_list->size_of_struct
</PRE>
</CODE></BLOCKQUOTE>
The offset from the module start occasionally changes (before 2.4.0, it
was module.size_of_struct + 4), so it's a good idea to check the init and
cleanup addresses once in a while, as describe below.
Now do:
<BLOCKQUOTE><CODE>
<PRE>
(gdb) add-symbol-file /path/to/module/on/host that_address
</PRE>
</CODE></BLOCKQUOTE>
Tell gdb you really want to do it, and you're in business.</P>
<P>If there's any doubt that you got the offset right, like breakpoints
appear not to work, or they're appearing in the wrong place, you can
check it by looking at the module structure. The init and cleanup
fields should look like:
<BLOCKQUOTE><CODE>
<PRE>
init = 0x588066b0 <init_hostfs>, cleanup = 0x588066c0 <exit_hostfs>
</PRE>
</CODE></BLOCKQUOTE>
with no offsets on the symbol names. If the names are right, but they
are offset, then the offset tells you how much you need to add to the
address you gave to add-symbol-file.</P>
<P>
When you want to load in a new version of the module, you need to get
gdb to forget about the old one. The only way I've found to do that
is to tell gdb to forget about all symbols that it knows about:
<BLOCKQUOTE><CODE>
<PRE>
(gdb) symbol-file
</PRE>
</CODE></BLOCKQUOTE>
Then reload the symbols from the kernel binary:
<BLOCKQUOTE><CODE>
<PRE>
(gdb) symbol-file /path/to/kernel
</PRE>
</CODE></BLOCKQUOTE>
and repeat the process above. You'll also need to re-enable breakpoints.
They were disabled when you dumped all the symbols because gdb
couldn't figure out where they should go.</P>
<H2><A NAME="ss9.4">9.4</A> <A HREF="UserModeLinux-HOWTO.html#toc9.4">Using alternate debuggers</A>
</H2>
<P>UML has support for attaching to an already running debugger rather than
starting gdb itself. This is present in CVS as of
17 Apr 2001. I sent it to Alan for inclusion in
the ac tree, and it will be in my 2.4.4 release.</P>
<P>
This is useful when gdb is a subprocess of some UI, such as emacs or ddd. It
can also be used to run debuggers other than gdb on UML. Below is an example
of using strace as an alternate debugger.</P>
<P>
To do this, you need to get the pid of the debugger and pass it in with the
'gdb-pid=<pid>' switch along with the 'debug' switch.</P>
<P>
If you are using gdb under some UI, then tell it to 'att 1', and you'll find
yourself attached to UML.</P>
<P>
If you are using something other than gdb as your debugger, then you'll need
to get it to do the equivalent of 'att 1' if it doesn't do it automatically.</P>
<P>
An example of an alternate debugger is strace. You can strace the actual
kernel as follows:
<UL>
<LI>Run the following in a shell
<BLOCKQUOTE><CODE>
<PRE>
sh -c 'echo pid=$$; echo -n hit return; read x; exec strace -p 1 -o
strace.out'
</PRE>
</CODE></BLOCKQUOTE>
</LI>
<LI>Run UML with 'debug' and 'gdb-pid=<pid>' with the pid printed out by the
previous command
</LI>
<LI>Hit return in the shell, and UML will start running, and strace output will
start accumulating in the output file.
</LI>
</UL>
Note that this is different from running
<BLOCKQUOTE><CODE>
<PRE>
strace ./linux
</PRE>
</CODE></BLOCKQUOTE>
That will strace only the main UML thread, the tracing thread, which doesn't
do any of the actual kernel work. It just oversees the virtual machine. In
contrast, using strace as described above will show you the low-level activity
of the virtual machine.</P>
<HR>
<A HREF="UserModeLinux-HOWTO-10.html">Next</A>
<A HREF="UserModeLinux-HOWTO-8.html">Previous</A>
<A HREF="UserModeLinux-HOWTO.html#toc9">Contents</A>
</BODY>
</HTML>
|
