\documentclass[11pt]{article}
\usepackage{pl}
\usepackage{html}

\onefile
\htmloutput{html}				% Output directory
\htmlmainfile{index}				% Main document file
\bodycolor{white}				% Page colour

\renewcommand{\runningtitle}{SWI-Prolog C-library}

\begin{document}

\title{SWI-Prolog C-library}
\author{Jan Wielemaker \\
	SWI, \\
	University of Amsterdam \\
	The Netherlands \\
	E-mail: \email{jan@swi.psy.uva.nl}}

\maketitle

\begin{abstract}
This document describes commonly used foreign language extensions to
\url[SWI-Prolog]{http://www.swi.psy.uva.nl/projects/SWI-Prolog}
distributed as a package known under the name {\em clib}.  The package
defines a number of Prolog libraries with accompagnying foreign
libraries.

\begin{description}
    \item[\pllib{unix}]
This library provides Unix process control using fork(),
exec(), pipe(), etc.
    \item[\pllib{cgi}]
This library provides access to CGI form-data if Prolog is used for
CGI-scripting.
    \item[\pllib{crypt}]
This library provides access to Unix password encryption.
    \item[\pllib{mime}]
This library decodes MIME messages.
    \item[\pllib{socket}]
This library provides access to TCP/IP communication.
\end{description}

Currently only \pllib{socket} and \pllib{cgi} are available on
the default MS-Windows distribution.  The \pllib{unix} and
\pllib{crypt} libraries can be used if the whole SWI-Prolog suite
is compiled using \url[Cywin]{http://www.cygwin.com}.
\end{abstract}

\vfill

\pagebreak
\tableofcontents

\vfill
\vfill

\newpage

\section{Introduction}

Many useful facilities offered by one or more of the operating systems
supported by SWI-Prolog are not supported by the SWI-Prolog kernel
distribution.  Including these would enlarge the {\em footprint} and
complicate portability matters while supporting only a limited part of
the user-community.

This document describes \pllib{unix} to deal with the Unix process API,
\pllib{socket} to deal with inet-domain stream-sockets, \pllib{cgi} to
deal with getting CGI form-data if SWI-Prolog is used as a CGI scripting
language and \pllib{crypt} to provide access to Unix password
encryption.


\section{Unix Process manipulation library}

The \pllib{unix} library provides the commonly used Unix primitives to
deal with process management. These primitives are useful for many
tasks, including server management, parallel computation, exploiting and
controlling other processes, etc.

The predicates are modelled closely after their native Unix
counterparts. Higher-level primitives, especially to make this library
portable to non-Unix systems are desirable. Using these primitives and
considering that process manipulation is not a very time-critical
operation we anticipate these libraries to be developed in Prolog.

\begin{description}
    \predicate{fork}{1}{-Pid}
Clone the current process into two branches. In the child, \arg{Pid} is
unified to \const{child}. In the original process, \arg{Pid} is unified
to the process identifier of the created child.  Both parent and child
are fully functional Prolog processes running the same program.  The
processes share open I/O streams that refer to Unix native streams, such
as files, sockets and pipes. Data is not shared, though on most Unix
systems data is initially shared and duplicated only if one of the
programs attempts to modify the data.

Unix \funcref{fork}{} is the only way to create new processes and fork/2
is a simple direct interface to it.

    \predicate{exec}{1}{+Command(...Args...)}
Replace the running program by starting \arg{Command} using the given
commandline arguments. Each command-line argument must be atomic and is
converted to a string before passed to the Unix call \funcref{execvp}{}.

Unix \funcref{exec}{} is the only way to start an executable file
executing.  It is commonly used together with fork/1.  For example to
start \program{netscape} on an URL in the background, do:

\begin{code}
run_netscape(URL) :-
	(    fork(child),
	     exec(netscape(URL))
	;    true
	).
\end{code}

Using this code, netscape remains part of the process-group of the
invoking Prolog process and Prolog does not wait for netscape to
terminate.  The predicate wait/2 allows waiting for a child, while
detach_IO/0 disconnects the child as a deamon process.

    \predicate{wait}{2}{-Pid, -Status}
Wait for a child to change status. Then report the child that changed
status as well as the reason. \arg{Status} is unified with
\term{exited}{ExitCode} if the child with pid \arg{Pid} was terminated
by calling \funcref{exit}{} (Prolog halt/[0,1]). \arg{ExitCode} is the
return=status. \arg{Status} is unified with \term{signaled}{Signal} if
the child died due to a software interrupt (see kill/2). \arg{Signal}
contains the signal number. Finally, if the process suspended execution
due to a signal, \arg{Status} is unified with \term{stopped}{Signal}.

    \predicate{kill}{2}{+Pid, +Signal}
Deliver a software interrupt to the process with identifier \arg{Pid}
using software-interrupt number \arg{Signal}.  See also on_signal/2.
The meaning of the signal numbers can be found in the Unix manual.%
\footnote{kill/2 should support interrupt-names as well}.

    \predicate{pipe}{2}{-InSream, -OutStream}
Create a communication-pipe. This is normally used to make a child
communicate to its parent. After pipe/2, the process is cloned and,
depending on the desired direction, both processes close the end of the
pipe they do not use. Then they use the remaining stream to communicate.
Here is a simple example:

\begin{code}
:- use_module(library(unix)).

fork_demo(Result) :-
	pipe(Read, Write),
	fork(Pid),
	(   Pid == child
	->  close(Read),
	    format(Write, '~q.~n',
	           [hello(world)]),
	    flush_output(Write),
	    halt
	;   close(Write),
	    read(Read, Result),
	    close(Read)
	).
\end{code}

    \predicate{dup}{2}{+FromStream, +ToStream}
Interface to Unix dup2(), copying the underlying filedescriptor and thus
making both streams point to the same underlying object. This is
normally used together with fork/1 and pipe/2 to talk to an external
program that is designed to communicate using standard I/O.

Both \arg{FromStream} and \arg{ToStream} either refer to a Prolog stream
or an integer descriptor number to refer directly to OS descriptors.
See also \file{demo/pipe.pl} in the source-distribution of this package.

    \predicate{detach_IO}{0}{}
This predicate is intended to create Unix deamon-processes. It preforms
two actions.  First of all, the I/O streams \const{user_input}, 
\const{user_output} and \const{user_error} are closed and rebound to
a Prolog stream that returns end-of-file on any attempt to read and
starts writing to a file named \file{/tmp/pl-out.pid} (where <pid> is
the process-id of the calling Prolog) on any attempt
to write.  This file is opened only if there is data available.  This is
intended for debugging purposes.%
    \footnote{More subtle handling of I/O, especially for debugging is
	      required: communicate with the syslog deamon and
	      optionally start a debugging dialog on a newly created
	      (X-)terminal should be considered.}
Finally, the process is detached from the current process-group and its
controlling terminal.
\end{description}


\section{Socket library}

The \pllib{socket} library provides TCP inet-domain sockets from
SWI-Prolog, both client and server-side communication. The interface of
this library is very close to the Unix socket interface, also supported
by the MS-Windows {\em winsock} API.  Since SWI-Prolog 4.0, XPCE is part
of SWI-Prolog and offers \class{socket}.  XPCE provides an event-driven
interface to sockets, handling multiple open sockets in paralel.

In the future we hope to provide a more high-level socket interface
defined in Prolog and based on these primitives.

\begin{description}
    \predicate{tcp_socket}{1}{-SocketId}
Creates an \const{INET}-domain stream-socket and unifies an identifier
to it with \arg{SocketId}. On MS-Windows, if the socket library is not
yet initialised, this will also initialise the library.

    \predicate{tcp_close_socket}{1}{+SocketId}
Closes the indicated socket, making \arg{SocketId} invalid.  Normally,
sockets are closed by closing both stream handles returned by
open_socket/3.  There are two cases where tcp_close_socket/1 is used
because there are no stream-handles:

\begin{itemize}
    \item After tcp_accept/3, the server does a fork/1 to handle the client
	  in a sub-process. In this case the accepted socket is not
	  longer needed from the main server and must be discarded
	  using tcp_close_socket/1.
    \item If, after discovering the connecting client with tcp_accept/3,
          the server does not want to accept the connection, it should
	  discard the accepted socket immediately using tcp_close_socket/1.
\end{itemize}

    \predicate{tcp_open_socket}{3}{+SocketId, -InStream, -OutStream}
Open two SWI-Prolog I/O-streams, one to deal with input from the socket
and one with output to the socket.  If tcp_bind/2 has been called on the
socket. \arg{OutSream} is useless and will not be created. After closing
both \arg{InStream} and \arg{OutSream}, the socket itself is discarded.

    \predicate{tcp_bind}{2}{+Socket, +Port}
Bind the socket to \arg{Port} on the current machine.  This operation,
together with tcp_listen/2 and tcp_accept/3 implement the {\em server}-side of
the socket interface.

    \predicate{tcp_listen}{2}{+Socket, +Backlog}
Tells, after tcp_bind/2, the socket to listen for incoming requests for
connections.  \arg{Backlog} indicates how many pending connection
requests are allowed.  Pending requests are requests that are not yet
acknowledged using tcp_accept/3.  If the indicated number is exceeded, the
requesting client will be signalled that the service is currently not
available.  A suggested default value is 5.

    \predicate{tcp_accept}{3}{+Socket, -Slave, -Peer}
This predicate waits on a server socket for a connection request by
a client.  On success, it creates a new socket for the client and
binds the identifier to \arg{Slave}.   \arg{Peer} is bound to the
IP-address of the client.

    \predicate{tcp_connect}{+Socket. +Host:+Port}
Client-interface to connect a socket to a given \arg{Port} on a given
\arg{Host}.  After successful completion, tcp_open_socket/3 can be used to
create I/O-Streams to the remote socket.

    \predicate{tcp_setopt}{2}{+Socket, +Option}
Interface to setsockopt(), setting options for the socket.  Currently
the only defined option is \const{reuseaddr}, which allows servers to
reuse a port without the system being completely sure the port is no
longer in use.

    \predicate{tcp_fcntl}{3}{+Stream, +Action, ?Argument}
Interface to the Unix \funcref{fcntl}{} call.  Currently only suitable
to deal switch stream to non-blocking mode using:

\begin{code}
	...
	tcp_fcntlStream, setfl. nonblock),
	...
\end{code}

As of SWI-Prolog 3.2.4, handling of non-blocking stream is supported. An
attempt to read from a non-blocking stream returns -1 (or
\const{end_of_file} for read/1), but at_end_of_stream/1 fails. On actual
end-of-input, at_end_of_stream/1 succeeds.

    \predicate{tcp_host_to_address}{2}{?HostName, ?Address}
Translate between a machines host-name and it's (IP-)address. If
\arg{HostName} is an atom, it is resolved using
\funcref{gethostbyname}{} and the IP-number is unified to \arg{Address}
using a term of the format \term{ip}{Byte1, Byte2, Byte3, Byte4}.
Otherwise, if \arg{Address} is bound to a \functor{ip}{4} term, it
is resolved by \funcref{gethostbyaddr}{} and the canonical hostname
is unified with \arg{HostName}.

    \predicate{gethostname}{1}{-Hostname}
Return the official fully qualified name of this host.  This is achieved
by calling gethostname() followed by gethostbyname() and return the 
official name of the host (\const{h_name}) of the structure returned
by the latter function.
\end{description}


\subsection{Server applications}

The typical sequence for generating a server application is defined
below:

\begin{code}
create_server(Port) :-
	tcp_socket(Socket),
	tcp_bind(Socket, Port),
	tcp_listen(Socket, 5),
	tcp_open_socket(Socket, AcceptFd, _),
	<dispatch>
\end{code}

There are various options for <dispatch>. One is to keep track of active
clients and server-sockets using wait_for_input/3. If input arrives at a
server socket, use tcp_accept/3 and add the new connection to the active
clients. Otherwise deal with the input from the client.  Another is
to use (Unix) fork/1 to deal with the client in a separate process.

Using fork/1, <dispatch> may be implemented as:

\begin{code}
dispatch(AcceptFd) :-
	tcp_accept(AcceptFd, Socket, _Peer),
	fork(Pid)
	(   Pid == child
	->  tcp_open_socket(Socket, In, Out),
	    handle_service(In, Out),
	    close(In),
	    close(Out),
	    halt
	;   tcp_close_socket(Socket)
	),
	dispatch(AcceptFd).
\end{code}


\subsection{Client applications}

The skeleton for client-communication is given below.

\begin{code}
create_client(Host, Port) :-
	tcp_socket(Socket),
	tcp_connect(Socket, Host:Port),
	tcp_open_socket(Socket, ReadFd, WriteFd),
	<handle I/O using the two streams>
	close(ReadFd),
	close(WriteFd).
\end{code}

To deal with timeouts and multiple connections, wait_for_input/3 and/or
non-blocking streams (see tcp_fcntl/3) can be used.

\section{CGI Support library}

This is currently a very simple library, providing support for obtaining
the form-data for a CGI script:

\begin{description}
    \predicate{cgi_get_form}{1}{-Form}
Decodes standard input and the environment variables to obtain a list
of arguments passed to the CGI script.  This predicate both deals with
the CGI {\bf GET} method as well as the {\bf POST} method.  If the
data cannot be obtained, an \const{existence_error} exception is
raised.
\end{description}

Below is a very simple CGI script that prints the passed parameters.
To test it, compile this program using the command below, copy it to
your cgi-bin directory (or make it otherwise known as a CGI-script) and
make the query \verb$http://myhost.mydomain/cgi-bin/cgidemo?hello=world$

\begin{code}
% pl -o cgidemo --goal=main --toplevel=halt -c cgidemo.pl
\end{code}

\begin{code}
:- use_module(library(cgi)).

main :-
	cgi_get_form(Arguments),
	format('Content-type: text/html~n~n', []),
	format('<HTML>~n', []),
	format('<HEAD>~n', []),
	format('<TITLE>Simple SWI-Prolog CGI script</TITLE>~n', []),
	format('</HEAD>~n~n', []),
	format('<BODY>~n', []),
	format('<P>', []),
	print_args(Arguments),
	format('<BODY>~n</HTML>~n', []).

print_args([]).
print_args([A0|T]) :-
	A0 =.. [Name, Value],
	format('<B>~w</B>=<EM>~w</EM><BR>~n', [Name, Value]),
	print_args(T).
\end{code}

\subsection{Some considerations}

Printing an HTML document using format/2 is not really a neat way
of producing HTML.  A high-level alternative is provided by
\pllib{http/html_write} from the XPCE package.


\section{MIME decoding library}

MIME (Multipurpose Internet Mail Extensions) is a format for serializing
multiple typed data objects.  It was designed for E-mail, but it is also
used for other applications such packaging multiple values using the
HTTP POST request on web-servers.  Double Precision, Inc.\ has produced
the C-libraries rfc822 (mail) and rfc2045 (MIME) for decoding and
manipulating MIME messages. The \pllib{mime} library is a Prolog wrapper
around the rfc2045 library for deconding MIME messages.	

The general name `mime' is used for this library as it is anticipated to
add MIME-creation functionality to this message.

Currently the mime library defines one predicate:

\begin{description}
    \predicate{mime_parse}{2}{Data, Parsed}
Parse \arg{Data} and unify the result to \arg{Parsed}.  \arg{Data} is
one of:

\begin{description}
    \termitem{stream}{Stream}
Parse the data from \arg{Stream} upto the end-of-file.
    \termitem{stream}{Stream, Length}
Parse a maximum of \arg{Length} characters from \arg{Stream} or
upto the end-of-file, whichever comes first.
    \termitem{\arg{Text}}{}
Atoms, strings, code- and character lists are treated as valid
sources of data.
\end{description}

\arg{Parsed} is a tree structure of \term{mime}{Attributes, Data,
PartList} terms.  Currently either \arg{Data} is the empty atom
or \arg{PartList} is an empty list.%
	\footnote{It is unclear to me whether a MIME note can contain
		  a mixture of content and parts, but I believe the
		  answer is `no'.}
\arg{Data} is an atom holding the message data. The library
automatically decodes \jargon{base64} and \jargon{quoted-printable}
messages.  See also the \const{transfer_encoding} attribute below.

 \arg{PartList} is
a list of \funcref{mime}{3} terms.  \arg{Attributes} is a list holding
a subset of the following arguments.  For details please consult the
RFC2045 document.

\begin{description}
    \termitem{type}{Atom}
Denotes the Content-Type, how the \arg{Data} should be interpreted.
    \termitem{transfer_encoding}{Atom}
How the \arg{Data} was encoded.  This is not very interesting as the
library decodes the content of the message.
    \termitem{character_set}{Atom}
The character set used for text data. Note that SWI-Prolog's
capabilities for character-set handling are limited.
    \termitem{language}{Atom}
Language in which the text-data is written.
    \termitem{id}{Atom}
Identifier of the message-part.
    \termitem{description}{Atom}
Descrptive text for the \arg{Data}.
    \termitem{disposition}{Atom}
Where the data comes from.  The current library only deals with
`inline' data.
    \termitem{name}{Atom}
Name of the part.
    \termitem{filename}{Atom}
Name of the file the data should be stored in.
\end{description}
\end{description}

{\bf NOTE} This library is only built and installed if the maildrop
libraries are installed on your system.


\section{Unix password encryption library}

The \pllib{crypt} library defines crypt/2 for encrypting and testing
Unix passwords:

\begin{description}
    \predicate{crypt}{2}{+Plain, ?Encrypted}
This predicate can be used in three modes.  If \arg{Encrypted} is
unbound, it will be unified to a string (list of character-codes)
holding a random encryption of \arg{Plain}.  If \arg{Encrypted}
is bound to a list holding 2 characters and an unbound tail, these
two character are used for the \jargon{salt} of the encryption. Finally,
if \arg{Encrypted} is instantiated to an encrypted password the
predicate succeeds iff \arg{Encrypted} is a valid encryption of
\arg{Plain}.

\arg{Plain} is either an atom, SWI-Prolog string, list of characters
or list of character-codes.  It is not advised to use atoms, as this
implies the password will be available from the Prolog heap as a
defined atom.
\end{description}

\section{Memory files}

The \pllib{memfile} provides an alternative to temporary files, intended
for temporary buffering of data. Memory files in general are faster than
temporary files and do not suffer from security riscs or naming
conflicts associated with temporary-file management.  They do
assume proper memory management by the hosting OS and cannot be used
to pass data to external processes using a file-name.

There is no limit to the number of memory streams, nor the size of them.
However, memory-streams cannot have multiple streams at the same time
(i.e.\ cannot be opened for reading and writing at the same time).

These predicates are first of all intended for building higher-level
primitives.  See also sformat/3, atom_to_term/3, term_to_atom/2 and
the XPCE primitive pce_open/3.


\begin{description}
    \predicate{new_memory_file}{-Handle}
Create a new memory file and return a unique opaque handle to it.

    \predicate{free_memory_file}{+Handle}
Discard the memory file and its contents.  If the file is open it
is first closed.

    \predicate{open_memory_file}{+Handle, +Mode, -Stream}
Open the memory-file.  \arg{Mode} is currently one of \const{read}
or \const{write}.  The resulting handling is closed
using close/1.

    \predicate{size_memory_file}{+Handle, -Bytes}
Return the content-length of the memory-file it \arg{Bytes}.  The
file should be closed and contain data.

    \predicate{atom_to_memory_file}{+Atom, -Handle}
Turn an atom into a read-only memory-file containing the (shared)
characters of the atom.  Opening this memory-file in mode \const{write}
yields a permission error.

    \predicate{memory_file_to_atom}{+Handle, -Atom}
Return the content of the memory-file in \arg{Atom}.

    \predicate{memory_file_to_codes}{+Handle, -Codes}
Return the content of the memory-file as a list of character-codes
in \arg{Codes}.
\end{description}


\section{Installation}

\subsection{Unix systems}

Installation on Unix system uses the commonly found {\em configure},
{\em make} and {\em make install} sequence. SWI-Prolog should be
installed before building this package. If SWI-Prolog is not installed
as \program{pl}, the environment variable \env{PL} must be set to the
name of the SWI-Prolog executable. Installation is now accomplished
using:

\begin{code}
% ./configure
% make
% make install
\end{code}

This installs the foreign libraries in \file{$PLBASE/lib/$PLARCH} and
the Prolog library files in \file{$PLBASE/library}, where
\file{$PLBASE} refers to the SWI-Prolog `home-directory'.

\end{document}