1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
|
=head1 NAME
C<Devel::MAT::UserGuide> - a users' introduction to C<Devel::MAT>
=head1 OVERVIEW
The C<Devel::MAT> ecosystem allows developers of F<perl> programs to inspect
and analyse memory-related problems such as memory leaks, unexpected memory
consumption, or odd state. This is an "offline" analysis system, in the sense
that the analysis tools all run in a different process, possibly at a later
time, than the F<perl> process that is being analysed.
The basic workflow consists of two main stages: first a I<heap dump> file is
generated from the F<perl> process being debugged, at or around the time that
the problem becomes apparent, and secondly this file is loaded by an analysis
tool in order to inspect the contents.
These two stages are described here separately. It is important to note that
they don't have to be done at the same time, on the same F<perl> process, or
even on the same sort of machine. It is fully possible to capture the heap
from a process on, say, a small server, then copy the file to a development
workstation or laptop and analyse it there at a later time. It is for this
reason that the heap-dumping part, L<Devel::MAT::Dumper>, is now separated
into its own CPAN distribution. This means it can be installed on its own,
without all the extra dependencies the full set of analysis tools require.
=head1 CAPTURING THE HEAP
To generate the heap dump file that captures the contents of the heap, the
L<Devel::MAT::Dumper> module is used. Ultimately the C<dump> function within
it needs to be called, but usually one of the module load options can be used
on the F<perl> commandline to achieve this without requiring the running code
to be modified.
For example, the C<-dump_at_DIE> option means that a heap dump will be written
just before the process quits due to an uncaught exception:
$ perl -MDevel::MAT::Dumper=-dump_at_DIE program.pl
At this point, the program will start up and run normally, but if it is about
to die, it will first write a F<.pmat> file capturing the contents of the
memory.
...
Dumping to program.pl.pmat because of DIE
Can't call method "method" on an undefined value at program.pl line 123.
There are a variety of other options for other situations, to suit other sorts
of bugs and issues under investigation. For more options, see the
documentation at L<Devel::MAT::Dumper/IMPORT OPTIONS>.
=head1 ANALYSING THE HEAP
Now that we have a F<.pmat> file, we can load it and start to inspect the
contents. A lot of the smaller, simpler tools are built as plugins for the
main F<pmat> command shell, so we can start by loading the heap file there.
$ pmat program.pl.pmat
Perl memory dumpfile from perl 5.24.1
Heap contains 15624 objects
pmat>
In this shell a collection of commands is available to help analyse and
inspect the contents of memory represented by this heap dump, which can be
used in an interactive way, trying to narrow down to find the cause of the
memory issue.
It is hard in general to describe exactly what sequence of analysis commands
will be best to find the problem, as the specifics of each individual case
will call for different kinds of analysis and require us to ask different
questions of the toolset.
Ultimately there is quite a variety of possible underlying causes of memory
growth in a Perl program; a few possible causes could be:
=over 2
=item *
A single large SV such as a hash or array containing millions of items, or
a single string possibly gigabytes in length.
=item *
A large number of SVs being created retained indefinitely, never being
reclaimed.
=item *
A large number of temporary SVs being created, but due to internal reference
cycles their memory is never reclaimed despite them now being unreachable.
=back
This list of course is quite incomplete - there are as many different
variations of erroenous memory usage as there are possible programs to write.
Additionally, a lot of more interesting programs often suffer multiple
overlapping issues at once. Nevertheless, this broad categorisation can help
to describe some overall approaches to finding memory usage issues.
A good first step to take in the F<pmat> shell to try to distinguish these
cases is to use the C<largest> command. This command requires no additional
arguments, and by default will print (in size order), the five largest
individual SVs in the entire heap.
pmat> largest
For more information about the C<largest> command, see also
L<Devel::MAT::Tool::Sizes/largest>.
=head2 One Large SV
Sometimes you'll find a single SV that far outweighs all the others; for
example:
pmat> largest
SCALAR(PV) at 0x6a47708: 1.6 GiB
SCALAR(PV) at 0x1a59488: 4.0 MiB
HASH(0) at 0xfb4770=strtab: 1.5 MiB
SCALAR(PV) at 0x71b6468: 707.3 KiB
SCALAR(PV) at 0x71be2f0: 609.6 KiB
others: 46.2 MiB
In this output, we see that the topmost SV reported, at address C<0x6a47708>
is much larger than everything else put together. In this case we have
essentially already found the cause of the memory usage growth, and we can
proceed by identifying what this particular SV actually is, by following the
process in L<Devel::MAT::UserGuide::IdentifyingAnSV>.
For a brief overview, we can just count the total number of objects of various
kinds in the heap:
pmat> count
Kind Count (blessed) Bytes (blessed)
ARRAY 182 0 16.0 KiB
CODE 182 0 22.8 KiB
GLOB 325 0 48.2 KiB
...
We can inspect the callstack at the time the heap dump was made:
pmat> callers
caller(0): CODE(PP) at 0x55555582a4e8=&main::__ANON__ => void
at t/test.pl line 49
$_[0]: SCALAR(PV) at 0x55c2bdce2778 = "arguments"
$_[1]: SCALAR(PV) at 0x55c2bdce2868 = "go"
$_[2]: SCALAR(PV) at 0x55c2bdce26e8 = "here"
...
=head1 COMMAND HELP
A list of the commands currently available in the shell can be found by the
C<help> command:
pmat> help
callers - Display the caller stack
count - Count the various kinds of SV
elems - List the elements of an ARRAY SV
...
For more information about a particular command, give its name as an argument
to the C<help> command:
pmat> help sizes
sizes - Summarize object and byte counts across different SV types
SYNOPSIS:
sizes [OPTIONS...]
OPTIONS:
--owned sum SVs by owned size
--struct sum SVs by structural size
Also note that each command is implemented by a (correctly-cased) package
under the C<Devel::MAT::Tool> namespace. For example, the C<count> tool is
implemented by, and therefore more documentation can be found in, the
L<Devel::MAT::Tool::Count> package.
=head2 Specifying SVs
Many commands operate on a particular given SV. This can be specified in
several ways:
=over 4
=item *
A numerical address directly:
pmat> show 0x55a7e4e59f78
IO()=IO::File at 0x55a7e4e59f78 with refcount 1
...
=item *
A named root SV (see the C<roots> command for a list of them all):
pmat> show defstash
STASH(61) at 0x55a7e4d69060=defstash with refcount 2
=item *
A named symbol from the symbol table. Note that subs require the C<&> sigil:
pmat> show $warnings::VERSION
SCALAR(PV) at 0x55a7e4d96550 with refcount 1
...
pmat> show &warnings::import
CODE(PP) at 0x55a7e4dc3458 with refcount 1
...
=back
=head1 AUTHOR
Paul Evans <leonerd@leonerd.org.uk>
=cut
|
