1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246
|
package Munin::Node::SpoolReader;
use strict;
use warnings;
use Carp;
use IO::File;
use Fcntl qw(:DEFAULT :flock);
use Munin::Common::Defaults;
use Munin::Common::SyncDictFile;
use Munin::Node::Logger;
use Munin::Node::Config;
my $config = Munin::Node::Config->instance;
sub new
{
my ($class, %args) = @_;
$args{spooldir} or croak "no spooldir provided";
opendir $args{spooldirhandle}, $args{spooldir}
or croak "Could not open spooldir '$args{spooldir}': $!";
$args{metadata} = _init_metadata($args{spooldir});
# TODO: paranoia check? except dir doesn't (currently) have to be
# root-owned.
return bless \%args, $class;
}
#prepare tied hash for metadata persisted to $spooldir/SPOOL-META
#should we pull these methods into a base class or create a spool manager class?
sub _init_metadata
{
my $spooldir = shift;
my %metadata;
tie %metadata, 'Munin::Common::SyncDictFile', $spooldir . "/SPOOL-META";
return \%metadata;
}
#retrieve metadata value for key
sub get_metadata
{
my ($self, $key) = @_;
return ${ $self->{metadata} }{$key};
}
#set metadata key:value and persist
sub set_metadata
{
my ($self, $key, $value) = @_;
${ $self->{metadata} }{$key} = $value;
}
# returns all output for all services since $timestamp.
# The third argument ($submission_function) is optional: it is used for processing partial results,
# in order to avoid slurping the full data content into RAM. The result is empty, if this function
# is given.
sub fetch
{
my ($self, $timestamp, $submission_function) = @_;
my $return_str = '';
my @plugins = $self->_get_spooled_plugins();
logger("timestamp:$timestamp, plugins:@plugins") if $config->{DEBUG};
foreach my $plugin (@plugins) {
my $new_content = $self->_cat_multigraph_file(
$plugin, $timestamp, undef, $submission_function);
if (defined $submission_function) {
$submission_function->($new_content);
} else {
$return_str .= $new_content;
}
}
return $return_str;
}
sub list
{
my ($self) = @_;
my @plugins = $self->_get_spooled_plugins();
return join(" ", sort @plugins) . "\n";
}
# The last argument ($submission_function) is optional: it is used for processing partial results,
# in order to avoid slurping the full data content into RAM. The result is empty, if this function
# is given.
sub _cat_multigraph_file
{
my ($self, $service, $timestamp, $max_samples_per_service, $submission_function) = @_;
# Default $max_samples_per_service is 5, in order to have a 5x time
# increase in catchup. This enables to not overwhelm the munin-update when
# there is big backlog to handle. Use "0" to send an infinite number of
# samples
$max_samples_per_service = 5 if (! defined $max_samples_per_service);
my $data = "";
rewinddir $self->{spooldirhandle}
or die "Unable to reset the spool directory handle: $!";
my $nb_samples_sent = 0;
foreach my $file (sort readdir $self->{spooldirhandle}) {
# squash the $service name with the same rules as the munin-update when using plain TCP
# Closes D:710529
my $service_clean = $service;
$service_clean =~ s/[^_A-Za-z0-9]/_/g;
next unless $file =~ m/^munin-daemon\.$service_clean\.(\d+)\.(\d+)$/;
my $interval_start = $1;
my $interval_size = $2;
# skip the file if its newest value is older than the minimum wanted timestamp
next if $interval_start + $interval_size < $timestamp;
open my $fh, '<', "$self->{spooldir}/$file"
or die "Unable to open spool file: $!";
flock($fh, LOCK_SH);
my $epoch;
# wind through to the start of the first results after $timestamp
while (<$fh>) {
($epoch) = m/^timestamp (\d+)/ or next;
logger("Timestamp: $epoch") if $config->{DEBUG};
last if ($epoch > $timestamp);
}
if (eof $fh) {
logger("Epoch $timestamp not found in spool file for '$service'")
if $config->{DEBUG};
next;
}
# The timestamp isn't part of the multigraph protocol,
# just part of spoolfetch, so we have to filter it out,
# and replace each value line with its current value
while (<$fh>) {
chomp;
if (m/^timestamp (\d+)/) {
# epoch is updated
$epoch = $1;
next;
}
# Prepend the currently active timestamp, if neither a specific (numeric) nor an
# undefined ("N") epoch is included in the value line. The regular expression is
# supposed to match the following types of content:
# foo.value 12.3
# bar.value N:45.7
if (m/^(\w+)\.value\s+(?:N:)?([^:]+)$/) {
$_ = "$1.value $epoch:$2";
}
if (defined $submission_function) {
$submission_function->($_ . "\n");
} else {
$data .= $_ . "\n";
}
}
# We just emitted something
$nb_samples_sent ++;
if ($max_samples_per_service && $nb_samples_sent > $max_samples_per_service) {
logger("Already sent $nb_samples_sent for '$service', ending.") if $config->{DEBUG};
last;
}
}
return $data;
}
sub _get_spooled_plugins
{
my ($self) = @_;
rewinddir $self->{spooldirhandle}
or die "Unable to reset the spool directory handle: $!";
my %seen;
return map { m/^munin-daemon\.(.*)\.\d+\.\d+$/ && ! $seen{$1}++ ? $1 : () }
readdir $self->{spooldirhandle};
}
1;
__END__
=head1 NAME
Munin::Node::SpoolReader - Reading side of the spool functionality
=head1 SYNOPSIS
my $spool = Munin::Node::SpoolReader->new(spooldir => $spooldir);
print $spool->fetch(1234567890);
=head1 METHODS
=over 4
=item B<new($args)>
Constructor. 'spooldir' should be the directory L<Munin::Node::SpoolWriter> is
writing to.
=item B<fetch($timestamp)>
Fetches all the plugin results that have been recorded since C<$timestamp>,
in a form suitable to be sent straight over the wire.
=item B<list()>
Lists all the plugin that have been recorded in the spool, in a form suitable
to be sent straight over the wire.
=back
=cut
# vim: sw=4 : ts=4 : et
|