#  install.pod - Torrus installation instructions
#  Copyright (C) 2002-2011  Stanislav Sinyagin
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.

# Stanislav Sinyagin <ssinyagin@k-open.com>
#
#

=head1 Torrus Installation Instructions

B<Note to Debian users:> if you have installed torrus using Debian packages,
all the needed dependencies outlined below have been satisfied and 
installation tasks carried out by the Debian package management system.
The information below is provided for information purposes only and may
not apply to the Debian torrus installations. Please see the README.Debian
file in /usr/share/doc/torrus-common for more details.

=head2 Required Software

=over 4

=item * Operating System

Torrus should easily run on any UNIX-like operation system, as long as the
required components are provided.

If the OS mounts F</var/run> directory to a temporary filesystem, you
need t make sure that C<piddir> (F</var/run/torrus> by default) is
always available. If your OS uses Systemd, the Trrus distribution
supplies a I<tmpfile> file for that.

It is recommended to use Tobi Oetiker's SDBS tool available at
E<lt>https://github.com/oetiker/sdbsE<gt>. The tool compiles all Torrus
prerequisites from sources, and guarantees the version compatibility.


=item * Perl

Perl version C<5.8.9> or higher is required. Perl multithreading must be
enabled. Perl interpreter should be in PATH environment variable when
running C<./configure>, otherwise use

  ./configure PERL=/path/to/perl


=item * RRDtool

Round Robin Database Tool is available at:
E<lt>http://oss.oetiker.ch/rrdtool/E<gt>.

=item * Redis

A Redis database server is required for Torrus. By default Torrus is
configured to connect to port 6379 on 127.0.0.1, but this can be changed
in site configuration.

=item * libxml2

libxml2 is the XML parser library from Gnome
project E<lt>http://www.gnome.orgE<gt>,
available as a standalone package.
Versions C<2.4.23> and above were tested.

=item * HTTP Server

Torrus requires an HTTP server with FastCGI support for its front-end
UI. Typically Apache with mod_fcgid would be used.

See I<Torrus Web Interface Reference> for details on HTTP server configuration.

=item * Perl Modules

The Perl modules required are listed below. They are all available at
CPAN E<lt>http://www.cpan.orgE<gt>.
Some of them require other modules to be installed.

Provided that you have C<cpanminus> installed on your system, the
following command executed from within the distribution directory will
install all needed modules:

  sh setup_tools/cpanm_install_modules.sh

Modules required by Torrus:

=over 8

=item * Redis::Fast

=item * Git::Raw

=item * Git::ObjectStore

=item * Digest::SHA

=item * Cache::Ref

=item * JSON

=item * JSON::XS

=item * XML::LibXML

=item * Template

=item * Proc::Daemon

=item * Net::SNMP

=item * URI::Escape

=item * Apache::Session

=item * Date::Parse

=item * FCGI

=back

=back


=head2 Quick Installation with SDBS

  # This works in Debian or Ubuntu.
  # You need to install corresponding packages in orther systems
  apt-get update && apt-get install -y git gcc g++ make automake \
    libtool apache2 libapache2-mod-fcgid pkg-config xz-utils bzip2 \
    redis-server redis-tools

  mkdir -p /opt/pm/src
  cd /opt/pm/src

  git clone https://github.com/oetiker/sdbs.git
  cd sdbs
  PREFIX=/opt/pm/thirdparty ./build_perl5-current.sh
  PREFIX=/opt/pm/thirdparty ./build_rrdtool-current.sh

  echo 'PATH=/opt/pm/thirdparty/bin:/opt/pm/bin:$PATH; export PATH' \
    >/etc/profile.d/torrus_path.sh

  logout
  #login again

  # Watch out the group name, your OS may have a different group name for
  # Apache (this here works for Debian or Ubuntu)
  useradd -rU torrus
  usermod  -aG torrus www-data

  cd /opt/pm/src/
  git clone https://github.com/ssinyagin/torrus-newfeatures.git
  cd torrus-newfeatures/src
  sh setup_tools/cpanm_install_modules.sh
  autoreconf
  ./configure --prefix=/opt/pm piddir=/var/torrus/run
  make install

  # systemd unit files
  cp systemd/torrus.service /lib/systemd/system/
  cp systemd/torrus.tmpfile /usr/lib/tmpfiles.d/torrus.conf
  systemctl enable torrus

  # install plugins as necessary
  cd ../plugins/cbqos/
  autoreconf
  torrus install_plugin .

  mkdir /opt/pm/share/www
  vi /opt/pm/share/www/index.html
  === cut ===
  <html>
  <head>
  <title>redirect page</title>
  <META http-equiv="Refresh" CONTENT="0; URL=/torrus/">
  </head>
  <body></body>
  </html>
  === cut ===

  vi >/etc/apache2/sites-available/200-torrus.domain.com.conf
  === cut ===
  <VirtualHost *:80>
   DocumentRoot "/opt/pm/share/www"
   ServerName torrus.domain.com
   ScriptAlias /torrus "/opt/pm/torrus/bin/torrus.fcgi"
   FcgidMaxProcessesPerClass 10
   FcgidMaxRequestsPerProcess 100
   <Location />
     Options Indexes
     Require all granted
   </Location>
   <Location /torrus>
      Options       +ExecCGI
   </Location>
  </VirtualHost>
  === cut ===

  a2ensite 200-torrus.domain.com
  systemctl restart apache2

  mkdir -p /srv/torrus/collector_rrd
  chown torrus:torrus /srv/torrus/collector_rrd
  chmod g+s /srv/torrus/collector_rrd


=head2 Installation Procedure (in detail)

Create the group C<torrus> and a user C<torrus>. The user should be a member of
this group.

Add your Apache daemon user to the group C<torrus>.
Other users that will run any of Torrus processes must be included in
this group.

For example, in Solaris (you may need to specify the GID and UID numbers
according to your local administration policy):

  groupadd torrus
  useradd -c 'Torrus Daemon' -d /usr/local/torrus -g torrus torrus
  usermod -G www,torrus www

Further installation process is mostly as usual (watch out C<piddir> setting,
as described above):

  ./configure
  make
  make install

The default directory layout is described below. Should you need to change it,
there is a number of variables and configuration options that you may use
as C<./configure> arguments. See C<./configure --help> for details.
Alternatively you can utilize the script C<./setup_tools/configure_fhs>,
which is designed to provide an FHS compliant setup. The script is equivalent
to executing

  ./configure \
   --prefix=/opt \
   --mandir=/opt/share/man \
   pkghome=/opt/torrus \
   sitedir=/etc/opt/torrus

Do not try to change any paths by supplying them as C<make> variables,
this would most probably break the installation. The only C<make> variable
that is supported is C<DESTDIR>. It may be used for preparing the package for
further distribution. For example, the following command would install
all Torrus files as if F</tmp/stage> were the root of the filesystem:

  make DESTDIR=/tmp/stage install

The presence of prerequisite Perl modules is checked during the execution
of C<./configure>. You can disable this by giving I<--enable-pkgonly> option.

The installer sets up the site configuration files, but only if
they don't already exist.

Plugin modules should be installed separately, after the Torrus software is
successfully installed. For each plugin, unpack the plugin distribution archive
to some directory, and execute

  torrus install_plugin <UNPACKED_PLUGIN_DIR>

A number of directories is set up by default under the path C</var>,
and they must be writable by all Torrus processes, including the
Apache web server.

You can control these directories' access rights by setting the following
environment variables: I<var_user>, I<var_group>, I<var_mode>, like follows:

  ./configure var_group=wwwrun

Default values for operating systems other than Cygwin are: I<var_user=torrus>,
I<var_group=torrus>, I<var_mode=775>. Setgid bit is set by default for these
directories.

Attention for the systems that have F</var/run> in a temporary
directory. By default, Torrus installer creates a directory
F</var/run/torrus> to store the daemon PID files. After the server
reboot, the temporary filesystems are cleaned up, and the directory
would not exist any more. Use I<piddir=/var/torrus/run> or similar in
the ./configure arguments.


=head2 Logging

The collector and monitor daemons send their logs to a local SYSLOG
server in "local0" facility by default.

The SYSLOG facility can be altered in F<torrus-siteconfig.pl>:

  $Torrus::Log::syslogFacility = 'local5';

Also the following two statements would revert to the old behavior (log
files written directly by the daemons). Please not that the daemons
would not rotate the logs on SIGHUP:

  $Torrus::Collector::useSyslog = 0;
  $Torrus::Monitor::useSyslog = 0;

If your system policy does not allow using the system SYSLOG daemon,
you can run your own (rsyslogd is recommended) and attach it to a UNIX
socket. In this case, the following statement in siteconfig would direct
all logging to an alternative SYSLOG socket:

  $Torrus::Log::syslogSockOpt = ['unix', '/home/jsmith/tmp/syslog.sock'];




=head2 Torrus directory layout

  @pkghome@/
        Home directory for Torrus distribution files

  @cfgdefdir@/
        torrus-config.pl and other configuration files

  @pkgbindir@/
        Command-line executables

  @docdir@/
        POD and TXT documentation files

  @exmpdir@/
        Miscelaneous example files

  @perllibdir@/
        Perl libraries

  @pluginsdir@/
        Plugins configuration

  @scriptsdir@/
        Scripts

  @supdir@/
        Supplementary files, DTDs, MIBs, color schemas, web plain files

  @tmpldir@/
        Renderer output templates

  @distxmldir@/
        Distrubution XML files

  @sitedir@/
        Site configurable files

  @siteconfdir@/
        Place for torrus-siteconfig.pl and other siteconfigs

  @siteconfdir@/discovery/
        Devdiscover input files

  @tmpluserdir@/
        User-defined Renderer output templates

  @sitexmldir@/
        User XML configuration files

  @mandir@
        Place for man pages. All articles have the prefix C<torrus_>

  @logdir@/
        Daemon logfiles

  @piddir@
        Daemon PID files

  @seslockdir@
  @sesstordir@
        Web interface session files

  @defrrddir@
        Recommended directory for RRD files generated by collectors



=head2 Configuring Torrus

The datasources are configured with C<%Torrus::Global::treeConfig>
hash in F<torrus-siteconfig.pl>.

In this hash, the keys give the tree names. The values for each tree name
are pointers to hashes, with the following keys and values:
I<xmlfiles> points to an array of source XML file names;
I<run> points to a hash with the names of the daemons
that would be automatically launched for the tree;
I<desription> gives a short line describing the tree contents.

Two additional arrays: C<@Torrus::Global::xmlAlwaysIncludeFirst> and
C<@Torrus::Global::xmlAlwaysIncludeLast> give a list of source XML
files that are included in every tree, in the beginning or in the end of
the XML files list. By default, this array consists of two files:
F<defaults.xml> and F<site-global.xml>. The second one is resided in
the user-configurable directory, and you can use it to override any
default settings.

Example:

  @Torrus::Global::xmlAlwaysIncludeFirst =
      ('defaults.xml', 'site-global.xml');

  %Torrus::Global::treeConfig = (
    'tree_A' => {
      'description' => 'The First Tree',
      'xmlfiles' => [qw(a1.xml a2.xml a3.xml)],
      'run' => { 'collector' => 1, 'monitor' => 1 } },
    'tree_B' => {
      'description' => 'The Second Tree',
      'xmlfiles' => ['b1.xml', 'b2.xml'],
      'run' => {} }
   );

XML files are read additively within each tree, in the order
as they are listed. The XML compiler searches for the files in several
directories, listed in C<@Torrus::Global::xmlDirs>. By default, this list
conssists of two paths, one for Torrus distribution files, and the other
for user files.

Files generated by C<devdiscover> usually contain I<include> statements
which add the vendor-specific XML files to the compilation.

Below follows a short description of some XML files that come with
Torrus distribution. Only F<site-global.xml> is installed in the directory
for user-configurable files, all others are placed in the distribution
directory.

=over 4

=item * defaults.xml

Default parameters for the root of the datasources tree.
Default view definitions. B<Note:> this file is automatically
overwritten by C<make install>.

=item * site-global.xml

Parameters that you want to change or define for your site.
This file will be compiled for every tree after F<defaults.xml>,
and this is the place to override the defaults. The file that is supplied
with the Torrus distribution has some useful parameters that you may simply
uncomment.
B<Note:> this file is never overwritten by C<make install>.

=item * snmp-defs.xml

SNMP collector defaults. The file defines several templates
used for collecting SNMP data.
Do not change this file.
You may redefine the needed parameters in your site-specific files
and templates.

=item * vendor/E<lt>vendorE<gt>.E<lt>productE<gt>[.E<lt>subsystemE<gt>].xml

SNMP collector definitions and templates for various hardware vendors and
products. C<devdiscover> includes some of these files automatically in the
configuration.

=item * generic/*.xml

SNMP collector definitions and templates for vendor-independent MIBs. Most
files are named after corresponding RFC numbers.

=back

In addition, the distribution package contains several example files.

For more details about XML configuration, see I<Torrus User Guide>
and I<Torrus XML Configuration Guide>.

=head2 Site configuration options

In addition to I<%Torrus::Global::treeConfig>, you may wish to set
some other parameters in your site configuration file
(F<torrus-siteconfig.pl>).

See F<torrus-config.pl> for the complete list
of varaibes that you may override in your site config. Among them,
most interesting are:

=over 4

=item * C<$Torrus::Renderer::companyName>

The text that you specify here will appear in the top left corner
of all HTML pages.

=item * C<$Torrus::Renderer::companyURL>

The company name text will be clickable with the URL specified in
this variable.

=back


=head2 Apache HTTP server configuration

See the I<Torrus Web Interface Reference> document for detailed instructions on
Apache configuration.


=head2 Access Control Lists

By default, Torrus web interface requires user authentication.
You can disable this by changing C<$Torrus::CGI::authorizeUsers>
to zero in your F<torrus-siteconfig.pl>.

ACLs are controlled by C<acledit> utility. See I<Torrus Manual pages>
for detailed description. Example:

  torrus acledit --addgroup=staff --permit=DisplayTree --for='*'
  torrus acledit --addgroup=staff --permit=GlobalSearch --for='*'
  torrus acledit --adduser=jsmith --password=mysecretpassword \
    --cn="John Smith" --addtogroup=staff
  torrus acledit --addgroup=admin \
    --permit=DisplayTree --permit=DisplayAdmInfo --for='*'


=head2 Cron Job

In order to clean old HTTP session data, it is recommended to run
F<cleanup> script in a cron job, once per day:

 #min hour mday month wday    who     command
 5    3    *    *     *       root    @pkgbindir@/cleanup


=head2 Startup script

The Torrus distriubution provides a System V init script which you
can install in the init scripts directory (/etc/init.d on most Unixes or
/usr/local/etc/rc.d/ on FreeBSD). The script C<torrus> is created during
the installation process in the subdirectory <init.d> of the
distribution directory.

The init script reads some parameters from F<@cfgdefdir@/initscript.conf>,
and F<@siteconfdir@/initscript.siteconf> if the latter exists.
By default, it makes the monitor daemons sleep for 20 minutes when
they are launched simultaneously with collector daemons.

Also Torrus distribution comes with Systemd and Tmpfiles files, so if
the OS has Systemd in place, System V init files should not be used. See
the quick installation section (above) for more details.



=head2 Upgrading from Torrus version 2.x

Torrus 3.x is using a completely different back-end for its data
structures: the old Torrus was using BerkeleyDB, while the new one uses
a Git repository. Also run-time data is kept in a Redis database
instance.

Torrus 3.0 is fully compatible with XML configuration and
F<torrus-siteconfig.pl> statements of Torrus 2.0.

The best approach in upgrading from version 2 is to delete the old
database and cache directories (or simply everything in F</var/torrus>),
and also the Torrus scripts and libraries (F</opt/pm/torrus> if you used
F</opt/pm> as a prefix). Then, run the installation procedure and
execute the XML compiler, and Torrus should be ready to run.



=head2 Upgrading within version 3.x

Follow these instructions when upgrading from previous Torrus version.

In the previous distribution directory, look up the F<config.log> file.
It contains the configure options that you used in previous installation.

Unpack the new release distribution.

Run C<./configure> with the same options you used before.

Stop the collector and monitor processes (usually by
C</etc/init.d/torrus stop> or C<systemctl stop torrus>).

Stop Apache process.

Run C<make install>.

Start the collector and monitor processes.

Start Apache process.

Also it is recommended to re-compile your XML configuration.


=head2 Plugins

Plugin packages inherit a release numbering scheme as follows: a plugin
release version is a 3-component number, where the first two numbers are
the Torrus release version that is compatible with this plugin.


=head2 Planning the disk space

In a typical Torrus setup, there are two disk space consuming entities:
the RRDtool data storage and the Git repository for configuration data.

RRDtool data files (or RRD's) are used to store the values that are gathered
by the collector daemons. These are the most intensively updated files,
so the disk speed is important here. If possible, it is better to spread
the RRD files across several physical disks.

The Git repository is used to keep all the configuratiuon data for your
datasource trees, and it also keeps some live status information.  It's
updated during XML compilation only.  During normal Torrus working
cycle, there are only read operations. The database is read quite
intensively during collector initialization, but usually it's the CPU
speed that causes more delay.

The Git repository needs double the space it occupies between the
compilations. The compiler runs the garbage collector at the end of its
work, and all unnecessary data is deleted.


=head2 Network performance tuning

In large installations, the SNMP collector experiences quite intensive
input traffic bursts. Thus the UDP socket receive buffers should
be adapted to sustain the load. By default, Torrus sets the UDP receiving
buffer size, SO_RCVBUF, to 131071 bytes. This should fit most of
installations. However, it's useful to check the network statistics
if there are any UDP buffer overflows. On most systems, the commands
C<netstat -s -p udp> or C<netstat -s> should show you these counters.
The maximum buffer size is usually limited by a system kernel variable,
and can be increased if needed. See C<$Torrus::Collector::SNMP::RxBuffer>
and its comments in F<torrus-config.pl> for more details.


=head1 Author

Copyright (c) 2002-2017 Stanislav Sinyagin E<lt>ssinyagin@k-open.comE<gt>