1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
|
<?xml version='1.0'?>
<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
]>
<section id="dropwatchsect">
<title>Monitoring Network Packets Drops in Kernel</title>
<indexterm>
<primary>script examples</primary>
<secondary>network profiling</secondary>
</indexterm>
<indexterm>
<primary>examples of SystemTap scripts</primary>
<secondary>network profiling</secondary>
</indexterm>
<indexterm>
<primary>network profiling</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<indexterm>
<primary>profiling the network</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<indexterm>
<primary>network traffic, monitoring</primary>
<secondary>examples of SystemTap scripts</secondary>
</indexterm>
<para>
<indexterm><primary>tracepoint</primary></indexterm>
The network stack in Linux
can discard packets for various reasons. Some Linux kernels include a
tracepoint, <command>kernel.trace("kfree_skb")</command>, which easily tracks where packets are discarded.
<xref linkend="dropwatch"/> uses
<command>kernel.trace("kfree_skb")</command> to trace
packet discards; the script summarizes which locations
discard packets every five-second interval.
</para>
<formalpara id="dropwatch">
<title>dropwatch.stp</title>
<para>
<programlisting><xi:include parse="text" href="../testsuite/systemtap.examples/network/dropwatch.stp" xmlns:xi="http://www.w3.org/2001/XInclude" /></programlisting>
</para>
</formalpara>
<para>
The <command>kernel.trace("kfree_skb")</command> traces which places
in the kernel drop network packets.
The <command>kernel.trace("kfree_skb")</command> has two arguments:
a pointer to the buffer being freed (<command>$skb</command>)
and the location in kernel code the
buffer is being freed (<command>$location</command>).
The <xref linkend="dropwatch"/> script provides the function containing
<command>$location</command> where possible.
The information to map <command>$location</command> back to the
function is not in the instrumentation by default.
On SystemTap 1.4 the <command>--all-modules</command> option will include the
required mapping information and the following command can be used to run
the script:
</para>
<programlisting>stap --all-modules dropwatch.stp</programlisting>
<para>
On older versions of SystemTap you can use the following command to emulate the
<command>--all-modules</command> option:
</para>
<programlisting>stap -dkernel \
`cat /proc/modules | awk 'BEGIN { ORS = " " } {print "-d"$1}'` \
dropwatch.stp</programlisting>
<para>
Running the dropwatch.stp script 15 seconds would result in output similar in
<xref linkend="dropwatchoutput"/>. The output lists the number of misses for
each tracepoint location with either the function name or the address.
</para>
<example id="dropwatchoutput">
<title><xref linkend="dropwatch"/> Sample Output</title>
<screen>Monitoring for dropped packets
Tue Nov 17 00:26:51 2020
1762 packets dropped at unix_stream_recvmsg
4 packets dropped at tun_do_read
2 packets dropped at nf_hook_slow
Tue Nov 17 00:26:56 2020
467 packets dropped at unix_stream_recvmsg
20 packets dropped at nf_hook_slow
6 packets dropped at tun_do_read
Tue Nov 17 00:27:01 2020
446 packets dropped at unix_stream_recvmsg
4 packets dropped at tun_do_read
4 packets dropped at nf_hook_slow
Stopping dropped packet monitor
</screen>
</example>
<para>
When the script is being compiled on one machine and run on
another the <command>--all-modules</command> and
<filename>/proc/modules</filename> directory are not available; the
<command>symname</command> function will just print out the raw address.
To make the raw address of packet drops more meaningful, refer to the
<filename>/boot/System.map-`uname -r`</filename> file. This file lists the
starting addresses for each function, allowing you to map the
addresses in the output of <xref linkend="dropwatchoutput"/> to a specific
function name.
Given the following snippet of the
<filename>/boot/System.map-`uname -r`</filename> file,
the address 0xffffffff8149a8ed maps to the function
<command>unix_stream_recvmsg</command>:
</para>
<screen>[...]
ffffffff8149a420 t unix_dgram_poll
ffffffff8149a5e0 t unix_stream_recvmsg
ffffffff8149ad00 t unix_find_other
[...]</screen>
</section>
|
