.\" 
.\" Man page for pipexec
.\"
.\" Copyright 2015,2022 by Andreas Florath
.\" For license, see the 'LICENSE' file.
.\" SPDX-License-Identifier: GPL-2.0-or-later
.\"
.TH pipexec 1 2022-07-18 "User Commands" "User Commands"
.SH NAME
pipexec \- create a directed graph of processes and pipes
.SH SYNOPSIS
pipexec [OPTION]... [PROCESS DESCRIPTION]... [PIPE DESCRIPTION]...
.SH DESCRIPTION
.B pipexec
creates an arbitrary network (directed graph) of processes and pipes
in between - even cycles are possible.  It overcomes the shortcomings
of shells that are typically only able to create non cyclic trees.
.P
.B pipexec
also monitors all it's child processes and is able to restart the
whole network of processes and pipes if one crashes.
Therefore
.B pipexec
can be used in SYSV-init or systemd configuration to run a network of
processes.
.SH OPTIONS
.TP
\fB\-h\fR
print help and version information
.TP
\fB\-l logfd\fR
use the given file descriptor for text logging.  If a 's' is specified,
syslog is used.  Example: Specifying '2' means log to stderr.
.TP
\fB\-j logfd\fR
use the given file descriptor for json logging.  If a 's' is specified,
syslog is used.  Example: Specifying '2' means log to stderr.
As this is meant to be parsed by other programs, this is an official and
supported interface which is described in the JSON LOGGING chapter.
.TP
\fB\-p pidfile\fR
with
.B pipexec
it is possible to handle pipes within SYSV-init scripts.  In some
environments (e.g. RHEL6, Debian7) the start and stop routines need a
pid file.  If this option is given, pipexec writes its own pid into the
file shortly after start of pipexec.
.TP
\fB\-k\fR
if one sub-process (child) gets killed and this options is given, all
other sub-processes are also killed.  Afterwards all processes are
restarted.
.TP
\fB\-s sleep_time\fR
the time interval in seconds before a restart.  This option makes only
sense when also the '\-k' option is specified.
.SH BACKGROUND
Inside a shell it is possible to start processes and redirect the
output to other processes.
.P
Example:
.nf
    cat Chap1.txt Chap2.txt | grep bird | wc \-l
.fi
.P
Three processes are created: the standard output (file descriptor (fd)
1) of the 'cat' process is connected to the standard input (fd 0) of
the 'grep' command, and the standard output of the 'grep' command is
connected to the standard input (fd 0) of the 'wc' process.
.P
Please note that the assignment between names and file descriptor
number is pure historical and has no technical background.
.P
Example:
.nf
    find / 1> >(grep .txt) 2> >(wc >/tmp/w.log)
.fi
.P
In this more complex example, the fd 1 of the 'find' process is
connected to fd 0 of 'grep' and fd 2 is connected to fd 0 of 'wc'.
.P
The limitation using this way of specifying processes and pipes is,
that it is not possible to have any cycles.  It is impossible to
e.g. pass a fd of 'wc' either to 'grep' or to 'find'.
.P
.B pipexec
overcomes these limitations.  It makes it possible to link any two
arbitrary file descriptors in a set of processes.
.SH USAGE
When building up a network of processes and pipes, there is the need to
specify each element separately.
.P
The processes will be the nodes in the network (directed graph), the
connections of the file descriptors between to processes are the
edges.  Each node (process) has a unique name assigned to it.  This
makes it possible to differentiate between using the same command more
than once.
.P
The format of specifying a process is
.nf
    [ NAME /path/to/command arg1 arg2 ... argN ]
.fi
.P
The first parameter 'NAME' must be a unique name.  The second
parameter must be the full path of the command to execute.  Please
note that always the full path must be specified, there is no PATH
environment variable handling (execv(2) is used internally to span new
processes).  The following parameters are the parameters passed to the
command.
.P
The whole definition must be enclosed in square brackets.  The square
brackets must be given separately - before and after them must be a
space.
.P
The format of specifying a pipe between processes is
.nf
    {NAME_1:FD1>NAME_2:FD2}
.fi
.P
Example
.nf
    {LS:1>GREP:0}
.fi
.P
The names are the names of the processes, the numbers are the number
of the file descriptor that should be used to build the pipe in
between.  When using pipexec from a shell (like bash) there is the
need to escape the brackets or use quotation marks.
.SH JSON LOGGING
.B pipexec
can log in JSON format. This is an official supported interface which is
defined in this chapter. This can be seen as stable as no changes will
be made during small version upgrades. Of course additions will be made
also within minor upgrades, but this should be fine because of the underlaying
JSON format.
.P
Each JSON log object (e.g. line) will be in a separate line.
.P
The following keys are always present:
.TP
\fBtimestamp\fR
The timestamp is the time(0) (seconds since epoch) when the log in generated.
.TP
\fBpipexec_pid\fR
The pid of the pipexec process.
.TP
\fBid\fR
An id that specifies defined log objects.
.TP
\fBtype\fR
An indicator in which internal module this log was generated.
.TP
\fBserverity\fR
One of debug, info, warning, error.
.TP
\fBmessage\fR
A short message describing the event.
.P
Depending on the id there might be additional fields which are described
in the next section.
.TP
\fBid = 0\fR
id of 0 is a special case: this is used for internal logs only. The
logs are not documented and may change at any time. The information
can be used to get an idea what currently happens in pipexec.
.TP
\fBid = 1\fR
This gives information about the process ids (pids) of the commands.
The field \fBcommand\fR contains the command (i.e. the part in the command
before the colon. The field \fBcommand_pid\fR contains the pid of the
command.
.TP
\fBid = 2\fR
This log message is emitted when a child (command) exits. It contains some
information about the exit status and termination reason of the command.
The field \fBcommand_pid\fR contains the pid of the command which just
terminated. The field \fBstatus\fR is the value which is set by
\fBwaitpid(2)\fR. \fBnormal_exit\fR, \fBchild_status\fR and
\fBchild_signaled\fR are \fBWIFEXITED\fR, \fBWEXITSTATUS\fR and
\fBWIFSIGNALED\fR of the status respectively.
.SH RETURN
pipexec returns 1 if any of the child processes fails else 0 is
returned.
.SH EXAMPLES
The shell command
.nf
    cat Chap1.txt Chap2.txt | grep bird | wc \-l
.fi
.P
is equivalent to
.nf
    pipexec [ CAT /bin/cat Chap1.txt Chap2.txt ] \\
      [ GREP /usr/bin/grep bird ] [ WC /usr/bin/wc \-l ] \\
      "{CAT:1>GREP:0}" "{GREP:1>WC:0}"
.fi
.P
The pipexec equivalent is longer and more complex in this example.
But pipexec can build cycles that are impossible within a shell:
.nf
    pipexec [ A /bin/cmd1 ] [ B /bin/cmd2 ] "{A:1>B:0}" "{B:1>A:0}"
.fi
.P
When using json log, you get output like:
.nf
{"timestamp":1655706460,"pipexec_pid":42850,"id":1,"type":"exec","serverity":"info","message":"New child forked","command":"A","command_pid":"42851"}
{"timestamp":1655706886,"pipexec_pid":42869,"id":2,"type":"tracing","serverity":"info","message":"child exit","command_pid":"42870","status":"1","normal_exit":"1","child_status":"1","child_signaled":"0"}
.fi
.P
For more examples see the ptee(1) and peet(1) man pages.
.SH "SEE ALSO"
.BR bash(1),
.BR ptee(1),
.BR peet(1),
.BR execv(2)
.SH AUTHOR
Written by Andreas Florath (andreas@florath.net)
.SH COPYRIGHT
Copyright \(co 2015,2022 by Andreas Florath (andreas@florath.net).
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.