General documentation for the parallel package for Octave.

   Copyright (C) 2016-2023 <Olaf Till <i7tiol@t-online.de>>

   You can redistribute this documentation and/or modify it under the
terms of the GNU General Public License as published by the Free
Software Foundation; either version 3 of the License, or (at your
option) any later version.

   This documentation is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
Public License for more details.

   You should have received a copy of the GNU General Public License
along with this documentation; if not, see
<http://www.gnu.org/licenses/>.

General documentation for the parallel package for Octave
*********************************************************

This documentation applies to version 4.0.2 of the parallel package.

   The package contains functions for explicit local parallel execution
and functions for parallel execution over a cluster of machines,
possibly combined with local parallel execution at each machine of the
cluster.

   Earlier versions of the package used Octave sessions directly
generated by the 'fork' system call.  Since this causes problems, the
current version uses newly started Octave sessions, even for local
parallel execution.  This causes some limitations, see
Limitations.

   The server for cluster execution is installed at GNU/Linux, but not
at some other operating systems, e.g.  not at Windows or macOS. The
client functions for cluster execution should be installed at all
systems, though.

Indices

1 Installation hints
********************

_Note:_ For using a cluster of machines, identical versions of the
package are required to be installed at each machine.

   Maybe your distribution provides the parallel package.

   If not, or if you want a newer version, Octaves 'pkg' command allows
building the package from source and installing it.  Note that you have
to 'load' any Octave package before you can use it.  See Octaves
documentation of 'pkg'.

   Parallel execution over a cluster by default is protected by
encryption with authentication.  This needs a version of the gnutls
library in which TLS-SRP is not disabled.  For building the parallel
package from source, this library must be present during the build; some
operating systems provide development files of libraries in exta
'...-dev' packages, which are also needed during the build.

   Building or using the package for cluster execution without gnutls
will work, but encryption and authentication are not available in this
case.

2 Functions for local parallel execution
****************************************

Explicit local parallel execution, with the intent to exploit more than
one local processor(-core), is performed by calling a user-defined
function in parallel with several different arguments.  This is done in
parallel _processes_.  Note that there is no way to pass global
variables to or from these processes.

   The interface of the functions for local parallel execution is
similar to Octaves 'cellfun' and 'arrayfun' functions.

   Note that some operations in Octave, particularly some matrix
operations, may already be performed in parallel threads.  This may
limit the advantage yielded by explicit local parallel execution.  Also,
RAM access can be a bottleneck which limits computation speed of
multicore computing.

   For limitations, see Limitations.

2.1 Function parcellfun
=======================

 -- Function File: [O1, O2, ...] = parcellfun (NPROC, FUN, A1, A2, ...)
 -- Function File: parcellfun (nproc, fun, ..., "UniformOutput", VAL)
 -- Function File: parcellfun (nproc, fun, ..., "ErrorHandler", ERRFUNC)
 -- Function File: parcellfun (nproc, fun, ..., "VerboseLevel", VAL)
 -- Function File: parcellfun (nproc, fun, ..., "ChunksPerProc", VAL)
 -- Function File: parcellfun (nproc, fun, ..., "CumFunc", CUMFUNC)
     Evaluates a function for multiple argument sets using multiple
     processes in parallel.

     NPROC specifies the number of processes and must be at least 1.  It
     will be cut to the number of available CPU cores or the size of the
     input argument(s), whichever is lower.  FUN is a function handle
     pointing to the requested evaluating function.  A1, A2 etc.  should
     be cell arrays of equal size.  O1, O2 etc.  will be set to
     corresponding output arguments.

     The UniformOutput and ErrorHandler options are supported with
     meaning identical to "cellfun".  If option VerboseLevel is set to 1
     (default is 0), progress messages are printed to stderr.  The
     ChunksPerProc option controls the number of chunks which contains
     elementary jobs.  This option is particularly useful when time
     execution of function is small.  Setting this option to 100 is a
     good choice in most cases.

     Instead of returning a result for each argument, parcellfun returns
     only one cumulative result if "CumFunc" is non-empty.  CUMFUNC must
     be a function which performs an operation on two sets of outputs of
     FUN and returnes as many outputs as FUN.  If NOUT is the number of
     outputs of FUN, CUMFUNC gets a previous output set of FUN or of
     CUMFUNC as first NOUT arguments and the current output of FUN as
     last NOUT arguments.  The performed operation must be
     mathematically commutative and associative.  If the operation is
     e.g.  addition, the result will be the sum(s) of the outputs of FUN
     over all calls of FUN.  Since floating point addition and
     multiplication are only approximately associative, the cumulative
     result will not be exactly reproducible.

     Each process has its own pseudo random number generator, so when
     using this function for example to perform Monte-Carlo simulations
     one generally cannot expect results to be exactly reproducible.
     Exact reproducibility may be achievable by generating the random
     number sequence before calling parcellfun and providing the random
     numbers as arguments to FUN.

     See 'parallel_doc ("limitations")' for possible limitations on how
     FUN can be defined.

     See also: parcellfun_set_nproc.

2.2 Function pararrayfun
========================

 -- Function File: [O1, O2, ...] = pararrayfun (NPROC, FUN, A1, A2, ...)
 -- Function File: pararrayfun (nproc, fun, ..., "UniformOutput", VAL)
 -- Function File: pararrayfun (nproc, fun, ..., "ErrorHandler",
          ERRFUNC)
     Evaluates a function for corresponding elements of an array.
     Argument and options handling is analogical to 'parcellfun', except
     that arguments are arrays rather than cells.  If cells occur as
     arguments, they are treated as arrays of singleton cells.  Arrayfun
     supports one extra option compared to parcellfun: "Vectorized".
     This option must be given together with "ChunksPerProc" and it
     indicates that FUN is able to operate on vectors rather than just
     scalars, and returns a vector.  The same must be true for ERRFUNC,
     if given.  In this case, the array is split into chunks which are
     then directly served to FUNC for evaluation, and the results are
     concatenated to output arrays.  If "CumFunc" is also specified (see
     'parcellfun'), FUN is expected to return the result of the same
     cumulative operation instead of vectors.

   See also parcellfun, arrayfun (octave).

2.3 Function parcellfun_set_nproc
=================================

 -- Loadable Function: NPROC = parcellfun_set_nproc (NPROC_ARG)
     Set current number of background processes for parcellfun.

     The number of used background processes is determined by the first
     argument to 'parcellfun'.  If necessary, 'parcellfun' will spawn
     new background processes, but it will never remove any of them,
     even if it does not use all of them.  'parcellfun_set_nproc' can be
     used to set the number of currently spawned processes to NPROC_ARG.
     This may be used to remove all background processes between calls
     to 'parcellfun' by specifying an NPROC_ARG of zero.

     The real number of spawned processes will be limited to the
     available processor cores and is returned.

     See also: parcellfun.

3 Functions for parallel execution over a cluster of machines
*************************************************************

There are high level functions, similar to parcellfun and
pararrayfun, which only require the user to provide a function to be
called and a series of argument sets.  Parallel function calls are then
internally distributed over the cluster, and over the local processors
(or processor cores) within the single machines of the cluster.

   Medium and low level functions are also provided.  These require the
user to program the scheduling of parallel execution.

   Data transfer is possible between the client machine and each server
machine, but also directly between server machines.  The latter feature
is exploited e.g.  by the install_vars function.

   For limitations, see Limitations.

Connection-related functions

High level functions

Medium level functions

Low level functions

3.1 Security considerations
===========================

Over a connection to a server of the parallel package arbitrary Octave
commands can be executed at the server machine.  This means that any
change can be done to the server system and the resident data, only
limited by the rights of the system account under which the server runs.

   By default, the server is started with authentication and encryption
enabled, to avoid unauthorized access.  If the server is started with
authentication disabled (maybe to avoid the encryption overhead), it
must be cared for that no TCP connection by unauthorized persons is
possible to the server ports, possibly by running the client and all
server machines behind a firewall and assuring that only trusted persons
have access to any machine behind the firewall.  This scenario might be
achievable in home-nets.

   The server currently uses port 12502 for receiving commands and port
12501 for data exchange.

   The client and the servers used by the client with 'pconnect' must
agree on using authentication or not.

   Do not start the server as root.

3.2 Generating authentication keys
==================================

 -- Loadable Function: parallel_generate_srp_data (USERNAME)
 -- Loadable Function: parallel_generate_srp_data (USERNAME, OPTIONS)
     Prompts for a password (press enter for a random password) and
     writes TLS-SRP authentication files into the directory given by:

     'fullfile (a = pkg ("prefix"), "parallel-srp-data")'

     Server files are placed in subdirectory 'server'.  By default, a
     client authentication file is placed in subdirectory 'client'.  The
     latter contains the given USERNAME and the cleartext password.  You
     do not need this file if you prefer to be prompted for username and
     password at connection time.  In this case, you can prevent the
     client authentication file from being written by passing as the
     argument OPTIONS a structure with a value of 'false' in the field
     'unattended'.

     For authentication, subdir 'server', and possibly subdir 'client',
     have to be placed together with their contents at the respective
     machines (but see "Notes" below).  They can either be placed under
     the directory given by:

     'fullfile (OCTAVE_HOME (), "share", "octave", "parallel-srp-data")'

     or - which might be the same directory - under:

     'fullfile (a = pkg ("prefix"), "parallel-srp-data")'

     Files in the former directory will take precedence over those in
     the latter.  The contents of the files 'passwd' and 'user_passwd'
     (if present) must be kept secret.

     Notes: One of the two server files is named "passwd".  It contains
     one line per user - the line starts with the username and a ":".
     If this file pre-exists, the new line will be appended to it.  You
     may have to manually merge this file with a potentially
     pre-existing file at the server machine, e.g.  if you want to add a
     new user there.  Also, you have to manually assure that in the
     version of this file at the server machine each username appears
     only once (i.e.  old lines for the same user may have to be
     deleted).  The username is arbitrary.

     This function zeroizes sensitive data before releasing its memory.
     Due to usage of external libraries, however, it still can't be
     excluded that sensitive data is still on the swap device after
     application shutdown.

     See also: pconnect, pserver,
     reval, psend, precv,
     see sclose, select_sockets.

3.3 Starting servers
====================

 -- Function File: pserver ()
 -- Function File: pserver (OPTIONS)
 -- Function File: pserver ("kill")
     Starts or stops a server of the parallel cluster.

     OPTIONS: structure of options; field 'use_tls' is 'true' by default
     (TLS with SRP authentication); if set to 'false', there will be no
     encryption or authentication.  Field 'auth_file' can be set to an
     alternative path to the file with authentication information (see
     below).  Fields 'cmd_port' (default: 12502) and 'data_port'
     (default: 12501) can be set to change the ports of the command
     channel and the data channel, respectively.

     If called as 'pserver ("kill")', the server will be stopped by
     sending it a signal, taking its process id from its pid-file
     '/tmp/.octave-<hostname>.pid'.  Otherwise the server will be
     started.

     The servers exectuable file ('octave-pserver') is searched for by
     first assuming the directory structure of a regular package
     installation, then by searching Octaves function search path for
     it, and then by the called shell in its shell search path.

     If a directory path corresponding to the current directory of the
     client exists on the server machine, it will be used as the servers
     current directory for the respective client (multiple clients are
     possible).  Otherwise, '/tmp' will be used.  The Octave functions
     the server is supposed to call and the files it possibly has to
     access must be available at the server machine.  This can e.g.  be
     achieved by having the server machine mount a network file system
     (which is outside the scope of this package documentation).

     If a connection is accepted from a client, the server collects a
     network identifier and the names of all server machines of the
     network from the client.  Then, connections are automatically
     established between all machines of the network.  Data exchange
     will be possible between all machines (client or server) in both
     directions.  Commands can only be sent from the client to any
     server.

     The opaque variable holding the network connections, in the same
     order as in the corresponding variable returned by 'pconnect', is
     accessible under the variable name 'sockets' at the server side.
     Do not overwrite or clear this variable.  The own server machine
     will also be contained at some index position of this variable, but
     will not correspond to a real connection.  See 'pconnect' for
     further information.

     The client and the server must both use or both not use TLS. If TLS
     is switched off, different measures must be taken to protect ports
     of the command and data channels at the servers and the client
     against unauthorized access, e.g.  by a firewall or by physical
     isolation of the network.

     For using TLS, authorization data must be present at the server
     machine.  These data can conveniently be generated by
     'parallel_generate_srp_data'; the helptext of the latter function
     documents the expected location of these data.

     The SRP password will be sent over the encrypted TLS channel from
     the client to each server, to avoid permanently storing passwords
     at the server for server-to-server data connections.  Due to
     inevitable usage of external libraries, memory with sensitive data
     can, however, be on the swap device even after shutdown of the
     application.

     The server executable file 'octave-pserver' is installed and runs
     at GNU/Linux, but not at some other operating systems like Windows
     and macOS.

     See also: pconnect, reval,
     psend, precv, sclose,
     see parallel_generate_srp_data,
     see select_sockets.

3.4 Establishing cluster connections
====================================

 -- Loadable Function: CONNECTIONS = pconnect (HOSTS)
 -- Loadable Function: CONNECTIONS = pconnect (HOSTS, OPTIONS)
     Connects to a network of parallel cluster servers.

     As a precondition, a server must have been started at each machine
     of the cluster, see 'pserver'.  Connections are not guaranteed to
     work if client and server are from 'parallel' packages of different
     versions, so versions should be kept equal.

     HOSTS is a cell-array of strings, holding the names of all server
     machines.  The machines must be unique, and their names must be
     resolvable to the correct addresses also at each server machine,
     not only at the client.  This means e.g.  that the name 'localhost'
     is not acceptable (exception: 'localhost' is acceptable as the
     first of all names).

     Alternatively, but deprecated, HOSTS can be given as previously, as
     a character array with a machine name in each row.  If it is given
     in this way, the first row must contain the name of the client
     machine (for backwards compatibility), so that there is one row
     more than the now preferred cell-array HOSTS would have entries.

     'pconnect' returns an opaque variable holding the network
     connections.  This variable can be indexed to obtain a subset of
     connections or even a single connection.  (For backwards
     compatibility, a second index of ':' is allowed, which has no
     effect).  At the first index position is the client machine, so
     this position does not correspond to a real connection.  At the
     following index positions are the server machines in the same order
     as specified in the cell-array HOSTS.  So in the whole the variable
     of network connections has one position more than the number of
     servers given in HOSTS (except if HOSTS was given in the above
     mentioned deprecated way).  You can display the variable of network
     connections to see what is in it.  The variable of network
     connections, or subsets of it, is passed to the other functions for
     parallel cluster execution ('reval', 'psend', 'precv', 'sclose',
     'select_sockets' among others - see documentation of these
     functions).

     OPTIONS: structure of options; field 'use_tls' is 'true' by default
     (TLS with SRP authentication); if set to 'false', there will be no
     encryption or authentication.  Field 'password_file' can be set to
     an alternative path to the file with authentication information
     (see below).  Field 'user' can specify the username for
     authentication; if the username is so specified, no file with
     authentication information will be used at the client, but the
     password will be queried from the user.

     The client and the server must both use or both not use TLS. If TLS
     is switched off, different measures must be taken to protect ports
     12501 and 12502 at the servers and the client against unauthorized
     access; e.g.  by a firewall or by physical isolation of the
     network.

     For using TLS, authorization data must be present at the server
     machine.  These data can conveniently be generated by
     'parallel_generate_srp_data'.  By default, the client
     authentication file is created in the same run.  The helptext of
     'parallel_generate_srp_data' documents the expected locations of
     the authentication data.

     The SRP password will be sent over the encrypted TLS channel from
     the client to each server, to avoid permanently storing passwords
     at the server for server-to-server data connections.  Due to
     inevitable usage of external libraries, memory with sensitive data
     can, however, be on the swap device even after shutdown of the
     application, both at the client and at the server machines.

     Example (let data travel through all machines), assuming 'pserver'
     was called on each remote machine and authentication data is
     present (e.g.  generated with 'parallel_generate_srp_data'):

          sockets = pconnect ({'remote.machine.1', 'remote.machine.2'});
          reval ('psend (precv (sockets(2)), sockets(1))', sockets(3));
          reval ('psend (precv (sockets(1)), sockets(3))', sockets(2));
          psend ('some data', sockets(2));
          precv (sockets(3))
          --> ans = some data
          sclose (sockets);

     See also: pserver, reval,
     psend, precv, sclose,
     parallel_generate_srp_data,
     select_sockets, rfeval.

3.5 Closing cluster connections
===============================

 -- Loadable Function: sclose (CONNECTIONS)
     Close the parallel cluster network to which CONNECTIONS belongs.

     See 'pconnect' for a description of the CONNECTIONS variable.  All
     connections of the network are closed, even if CONNECTIONS contains
     only a subnet or a single connection.

     See also: pconnect, pserver,
     reval, psend, precv,
     see parallel_generate_srp_data,
     see select_sockets.

3.6 Get network information
===========================

 -- Loadable Function: network_get_info (CONNECTIONS)
     Return an informational structure-array with one entry for each
     machine specified by CONNECTIONS.

     This function can only be successfully called at the client
     machine.  See 'pconnect' for a description of the CONNECTIONS
     variable.  CONNECTIONS can contain all connections of the network,
     a subset of them, or a single connection.  For the local machine
     (client), if contained in CONNECTIONS, some fields of the returned
     structure may be empty.

     The fields of the returned structure are 'local_machine': true for
     the connection representing the local machine, 'nproc': number of
     usable processors of the machine, 'nlocaljobs': configured number
     of local processes on the machine, 'peername': name of the machine
     (empty for local machine), 'open': true if the connection is open,
     'network_id': uuid of the network, 'real_node_id': internal id
     assigned to node, '0' for client, servers starting with '1'.

     See also: pconnect, pserver,
     reval, psend, precv,
     see sclose,
     parallel_generate_srp_data,
     select_sockets.

3.7 Set network properties
==========================

 -- Loadable Function: network_set (CONNECTIONS, KEY, VAL)
     Set the property named by KEY to the value VAL for each machine
     specified by CONNECTIONS.

     This function can only be successfully called at the client
     machine.  See 'pconnect' for a description of the CONNECTIONS
     variable.  CONNECTIONS can contain all connections of the network,
     a subset of them, or a single connection.

     Possible values of KEY: ''nlocaljobs'': configured number of local
     processes on the machine (usable by functions for parallel
     execution); needs a non-negative integer in VAL, '0' means not
     specified.

     See also: pconnect, pserver,
     reval, psend, precv,
     see sclose,
     parallel_generate_srp_data,
     select_sockets.

3.8 Function netcellfun
=======================

 -- Function File: netcellfun (CONNECTIONS, FUN, ...)
     Evaluates function FUN in a parallel cluster and collects results.

     This function handles arguments and options equivalently to
     'parcellfun' and returnes equivalent output.  Differently, the
     first argument specifies server machines for parallel remote
     execution, see 'pconnect' for a description of the CONNECTIONS
     variable.  A further difference is that the option "ChunksPerProc"
     is ignored and instead the chunk size can be specified directly
     with an option "ChunkSize".

     This function can only be successfully called at the client machine
     of the parallel cluster.  CONNECTIONS can contain all connections
     of the network, a subset of them, or a single connection.  The
     local machine (client), if contained in CONNECTIONS, is ignored.
     However, one of the servers can run at the local machine under
     certain conditions (see 'pconnect') and will not be ignored in this
     case, so that the local machine can take part in parallel execution
     with 'netcellfun'.

     As a second level of parallelism, FUN is executed at each server
     machine (using 'parcellfun or pararrayfun') by default in as many
     local processes in parallel as the server has processor cores
     available.  The number of local parallel processes can be
     configured for each server with the "nlocaljobs" option (see
     'network_set'), a value of '0' means that the default value will be
     used, a value of '1' means that execution is not parallel within
     the server (but still parallel over the cluster).

     See 'parallel_doc ("limitations")' for possible limitations e.g.
     on how FUN can be defined.

     Cluster execution incurs a considerable overhead.  A speedup is
     likely if the computation time of FUN is long.  To speed up
     execution of a large set of arguments with short computation times
     of FUN, increase "ChunkSize", possibly use "Vectorize" (see
     'pararrayfun'), and possibly experiment with increasing
     "nlocaljobs" from the default.

     See also: netarrayfun, pconnect,
     see pserver, sclose,
     rfeval, install_vars.

3.9 Function netarrayfun
========================

 -- Function File: netarrayfun (CONNECTIONS, FUN, ...)
     Evaluates function FUN in a parallel cluster and collects results.

     This function handles arguments and options equivalently to
     'pararrayfun' and returnes equivalent output.  Differently, the
     first argument specifies server machines for parallel remote
     execution, see 'pconnect' for a description of the CONNECTIONS
     variable.  A further difference is that the option "ChunksPerProc"
     is ignored and instead the chunk size can be specified directly
     with an option "ChunkSize" (option "Vectorized" can be used
     together with option "ChunkSize" in function 'netarrayfun').

     The further details of operation are the same as for 'netcellfun',
     please see there.

     See also: netcellfun, pconnect,
     see pserver, sclose,
     rfeval, install_vars.

3.10 Distribute Octave variables
================================

 -- Function File: install_vars (VARNAME, ..., CONNECTIONS)
     Install named variables at remote machines.

     The variables named in the arguments are distrubted to the remote
     machines specified by CONNECTIONS and installed there.  The
     variables must be accessible within the calling function.  If
     variables have been declared to have global scope, they will also
     have global scope at the remote machines.

     This function can only be successfully called at the client
     machine.  See 'pconnect' for a description of the CONNECTIONS
     variable.  CONNECTIONS can contain all connections of the network,
     a subset of them, or a single connection.  The local machine
     (client), if contained in CONNECTIONS, is ignored.

     To install, e.g., all visible variables,

     'install_vars (who (){:}, ...);'

     can be used.

     Internally, this function sends the variables only to one server
     and then issues the necessary commands to distribute them to all
     specified servers over server-to-server data connections.

     See also: pconnect, pserver,
     sclose, rfeval,
     netcellfun.

3.11 Single remote function evaluation
======================================

 -- Function File: rfeval (FUNC, ..., NOUT, ISOUT, CONNECTION)
     Evaluate a function at a remote machine.

     FUNC is evaluated with arguments '...' and number of output
     arguments set to NOUT at remote machine given by CONNECTION.  If
     ISOUT is not empty, it must be a logical array with NOUT elements,
     which are true for each of the NOUT output arguments which are
     requested from the function; the other output arguments will be
     marked as not requested with '~' at remote execution.

     This function can only be successfully called at the client
     machine.  See 'pconnect' for a description of the CONNECTION
     variable.  CONNECTION must contain one single connection.

     If an output argument is given to 'rfeval', the function waits for
     completion of the remote function call, retrieves the results and
     returns them.  They will be returned as one cell-array with an
     entry for each output argument.  If some output arguments are
     marked as not requested by setting some elements of ISOUT to false,
     the returned cell-array will only have entries for the requested
     output arguments.  For consistency, the returned cell-array can be
     empty.  To assign the output arguments to single variables, you can
     for example use: '[a, b, c] = returned_cell_array{:};'.

     If no output argument is given to 'rfeval', the function does not
     retrieve the results of the remote function call but returns
     immediately.  It is left to the user to retrieve the results with
     'precv'.  The results will be in the same format as if returned by
     'rfeval'.  Note that a cell-array, possibly empty, will always have
     to be retrieved, even if the remote function call should have been
     performed without output arguments.

     Parallel execution can be achieved by calling 'rfeval' several
     times with different specified server machines before starting to
     retrieve the results.

     The specified function handle can refer to a function present at
     the executing machine or be an anonymous function.  See
     'parallel_doc ("limitations")' for possible limitations.

     See also: pconnect, pserver,
     sclose, install_vars,
     netcellfun.

3.12 Sending Octave variables
=============================

 -- Loadable Function: psend (VALUE, CONNECTIONS)
     Send the value in variable VALUE to all parallel cluster machines
     specified by CONNECTIONS.

     This function can be called both at the client machine and (with
     'reval') at a server machine.  See 'pconnect' for a description of
     the CONNECTIONS variable, and 'pserver' for a description of this
     variable (named 'sockets') at the server side.  CONNECTIONS can
     contain all connections of the network, a subset of them, or a
     single connection.  The machine at which 'psend' was called (client
     or server), if contained in the CONNECTIONS variable, is ignored.

     The value sent with 'psend' must be received with 'precv' at the
     target machine.  Note that values can be sent to each machine, even
     from a server machine to a different server machine.

     If 'psend' is called at the client machine, a corresponding 'precv'
     should have been called before at the target machine, otherwise the
     client will hang if VALUE contains large data (which can not be
     held by the operating systems socket buffers).

     See also: pconnect, pserver,
     reval, precv, sclose,
     see parallel_generate_srp_data,
     see select_sockets.

3.13 Receiving Octave variables
===============================

 -- Loadable Function: precv (SINGLE_CONNECTION)
     Receive a data value from the parallel cluster machine specified by
     SINGLE_CONNECTION.

     This function can be called both at the client machine and (with
     'reval') at a server machine.  SINGLE_CONNECTION must be a single
     connection obtained by indexing the CONNECTIONS variable.  Please
     see 'pconnect' for a description of the CONNECTIONS variable, and
     'pserver' for a description of this variable (named 'sockets') at
     the server side.  If SINGLE_CONNECTION corresponds to the machine
     at which 'precv' was called, an error is thrown.

     The value received with 'precv' must be sent with 'psend' from
     another machine of the cluster.  Note that data can be transferred
     this way between each pair of machines, even sent by a server and
     received by a different server.

     If 'precv' is called at the client machine, a corresponding 'psend'
     should have been called before at the source machine, otherwise the
     client will hang.

     See also: pconnect, pserver,
     reval, psend, sclose,
     see parallel_generate_srp_data,
     see select_sockets.

3.14 Remote command evaluation
==============================

 -- Loadable Function: reval (COMMANDS, CONNECTIONS)
     Evaluate COMMANDS at all remote hosts specified by CONNECTIONS.

     This function can only be successfully called at the client
     machine.  COMMANDS must be a string containing Octave commands
     suitable for execution with Octaves 'eval()' function.  See
     'pconnect' for a description of the CONNECTIONS variable.
     CONNECTIONS can contain all connections of the network, a subset of
     them, or a single connection.  The local machine (client), if
     contained in CONNECTIONS, is ignored.  COMMANDS is executed in the
     same way at each specified machine.

     See also: pconnect, pserver,
     psend, precv, sclose,
     see parallel_generate_srp_data,
     see select_sockets.

3.15 Call Unix 'select' for data connections
============================================

 -- Loadable Function: select_sockets (CONNECTIONS, TIMEOUT)
 -- Loadable Function: select_sockets (CONNECTIONS, TIMEOUT, NFDS)
     Calls Unix 'select' for data connections in a parallel cluster.

     This function is for advanced usage (and therefore has minimal
     documentation), typically the programming of schedulers.  It can be
     called at the client or at a server.

     CONNECTIONS: valid connections object (see 'pconnect' and
     'pserver', possibly indexed).

     TIMEOUT: seconds, negative for infinite.

     NFDS: Passed to Unix 'select' as first argument, see documentation
     of Uix 'select'.  Default: 'FD_SETSIZE' (platform specific).

     An error is returned if nfds or a watched filedescriptor plus one
     exceeds FD_SETSIZE.

     Returns an index vector to CONNECTIONS indicating connections with
     pending input, readable with 'precv'.

     If called at the client, the command connections are included into
     the UNIX 'select' call and checked for error flags, and
     'select_sockets' returns an error if a flag for a remote error is
     received.

     See also: pconnect, pserver,
     reval, psend, precv,
     see sclose,
     parallel_generate_srp_data.

3.16 Example
============

     # From Octave prompt, generate authentication files, set user name to
     # 'test'. When prompted for a password, press <enter>.
     parallel_generate_srp_data ('test')

     # From Octave prompt, get location of the generated files.
     authpath = fullfile (a = pkg ("prefix"), "parallel-srp-data")

     Copy server files to servers, authpath is assumed to be
     "/home/test/octave/parallel-srp-data/", the same directory is assumed to
     exist on the servers. From the system shell, do e.g.:

     scp -r /home/test/octave/parallel-srp-data/server server1:/home/test/octave/parallel-srp-data/
     scp -r /home/test/octave/parallel-srp-data/server server2:/home/test/octave/parallel-srp-data/

     Start server at remote machines. From the system shell, do e.g.:

     ssh server1 'octave --eval "pserver"'
     ssh server2 'octave --eval "pserver"'

     # From Octave prompt, connect the cluster.
     conns = pconnect ({"server1", "server2"})

     # And perform some parallel execution. Single function calls take 1
     # second each.
     results = netcellfun (conns,  (x) {x, pause(1)}{:}, num2cell (1:30))

     # Close network.
     sclose (conns)

4 Functions possibly helpful in parallel execution
**************************************************

4.1 Transfer variable within the local machine over an Octave stream
====================================================================

 -- Loadable Function: fsave (FID, VAR)
     Save a single variable to a binary stream, to be subsequently
     loaded with fload.

 -- Loadable Function: VAR = fload (FID)
     Loads a single variable of any type from a binary stream, where it
     was previously saved with fsave.

4.2 Call Unix 'select' on Octave streams
========================================

Note that for cluster connections select_sockets should be used
instead.

 -- Loadable Function: [N, RIDX, WIDX, EIDX] = select (READ_FIDS,
          WRITE_FIDS, EXCEPT_FIDS, TIMEOUT[, NFDS])
     Calls Unix 'select', see the respective manual.

     The following interface was chosen: READ_FIDS, WRITE_FIDS,
     EXCEPT_FIDS: vectors of stream-ids.  TIMEOUT: seconds, negative for
     infinite.  NFDS: optional, default is Unix' FD_SETSIZE (platform
     specific).  An error is returned if nfds or a filedescriptor
     belonging to a stream-id plus one exceeds FD_SETSIZE. Return values
     are: N: number of ready streams.  RIDX, WIDX, EIDX: index vectors
     of ready streams in READ_FIDS, WRITE_FIDS, and EXCEPT_FIDS,
     respectively.

5 Limitations of high- and medium-level functions
*************************************************

Note that netcellfun, netarrayfun, and rfeval,
but even parcellfun and pararrayfun, execute code in
separate Octave processes.  This causes some limitations.

   For the function handles passed as arguments to parcellfun,
pararrayfun, netcellfun, netarrayfun, and
rfeval, the following limitations may apply:

   * In Octave versions before 5.1, anonymous functions may only be
     passed if they don't use 'varargin' and don't contain nested
     anonymous functions.
   * If a handle to a named function, written in a function file, is
     passed, this function must exist on the server machine.  This is
     usually no problem for parcellfun and pararrayfun.
   * Named functions defined in the command line interface or by scripts
     are not exported to the working sessions used by parcellfun
     or pararrayfun, and not automatically to the server for
     remote parallel execution.
   * For handles to subfunctions or to private functions limitations
     depending on the used Octave version may apply.

   Anonymous functions should be usable as usual, since the function
specification sent to the server will include the anonymous functions
context.

   If parcellfun or pararrayfun (and of course the
cluster-related functions, too) are called from within a function with
subfunctions, the subfunctions won't be visible in parallel execution.

   There is no way to set global variables for the function called by
'parcellun' or 'pararrayfun'.  For cluster execution, setting global
variables is possible, e.g.  with install_vars; however,
'netcellfun' and 'netarrayfun' won't 'see' global variables if they use
'parcellfun' internally (which is the default at remote machines with
more than one available processor core).

6 Function parallel_doc to view documentation
*********************************************

 -- Function File: parallel_doc ()
 -- Function File: parallel_doc (KEYWORD)
     Show parallel package documentation.

     Runs the info viewer Octave is configured with on the documentation
     in info format of the installed parallel package.  Without
     argument, the top node of the documentation is displayed.  With an
     argument, the respective index entry is searched for and its node
     displayed.