\documentclass[11pt,a4paper]{article} 
\usepackage[latin1]{inputenc} 
\usepackage[american]{babel}
\usepackage{verbatim}
\usepackage{url}
% %pour creer des lien vers des pages web
\usepackage[pdftex,colorlinks=true, urlcolor=cyan,pdfstartview=FitH]{hyperref}
%%pour inserer du code
\usepackage{listings}
\lstset{ %
         language=Python,
         showspaces=false,
         showstringspaces=false,
         showtabs=false,
         basicstyle=\footnotesize,
         }

%% pour afficher des images avec pdflatex utiliser le package pdftex
%\usepackage{graphicx}

%% MARGES
\oddsidemargin -1cm
\marginparwidth 0cm \textwidth 18.5cm
\topmargin -0.5cm
\headheight -0.8cm \headsep 0cm
%%\footskip 0cm
\textheight 26.8cm
\pagestyle{plain}

\date{ Mobyle 1.0 }

\title{ The Mobyle Execution System  }

\begin{document}
\maketitle

\tableofcontents

\section{concepts}
There are 3 main actors in the Mobyle execution system:
\begin{itemize}
\item The ExecutionSystem is the interface between Mobyle and the low level
execution system as your local system or your favorite Distributed Resources
Management System (DRMS),
\item The ExecutionConfig is an object which provides the basic information
needed by the ExecutionSystem to be used in your environment,
\item The Dispatcher chooses which execution system to use for a job.
\end{itemize}
Each actor is modelized as a class and several implementations are provided in
Mobyle distribution.

\subsection{The ExecutionSystem}
All execution System classes inherit from the abstract class ExecutionSystem
and are located in the MOBYLEHOME/Src/Mobyle/Execution package. 5 ExecutionSystem 
classes are available:
\begin{itemize}
  \item SYS which is the interface between Mobyle and your local system.
  \item SGE which is the interface between Mobyle and the Sun Grid Engine DRMS.
  \item SgeDRMAA which is the interface between Mobyle and the Sun Grid Engine
  DRMS using the drmaa library.
  \item PbsDRMAA which is the interface between Mobyle and the PBS/torque
  DRMS using the drmaa library. 
  \item Lsf DRMAA which is the interface between Mobyle and the LSF
  DRMS using the drmaa library.
\end{itemize}

For SGE and PBS, two execution systems are proposed: one which wraps
the shell commands and another which deals with Distributed Resource Management
Application Api (DRMAA) library. The benefits to use the DRMAA library is that
there is no need to use intermediate shell to run, get the status or
kill a job. 
For those who cannot or do not want to install libdrmaa, the legacy
SGE execution systems is kept.

\subsection{The ExecutionConfig}
Mobyle is highly flexible towards the execution of jobs, you can use several
Execution System in parallel. For instance you can use SGE for most of jobs and
SYS for some very small jobs or one cluster managed by SGE for a set of jobs and 
an other cluster managed by PBS for the other jobs on so on. 
For each execution system you want to use you must define an ExecutionConfig. 
this instanciation must be done in EXECUTION\_SYSTEM\_ALIAS. 
To each class of ExecutionSystem a class of ExecutionConfig is associated. So
the choice of an ExecutionConfig determines which ExecutionSystem will be
used.

\subsubsection{SgeDRMAAConfig}
is associated to SgeDRMAA ExecutionSystem, which is the interface with SGE
using libdrmaa. This class takes 3 mandatory parameters:
\begin{enumerate}
  \item the path of the drmaa library (eg. /usr/local/sge/lib/lx26-amd64/libdrmaa.so )
  \item root the content of the SGE\_ROOT variable.
  \item cell the content of the SGE\_CELL variable.
\end{enumerate}

\subsubsection{PbsDRMAAConfig}
is associated to PbsDRMAA ExecutionSystem, which is the interface with Pbs/torque
using libdrmaa. This class takes 2 mandatory parameters:
\begin{enumerate}
  \item the path of the drmaa library (eg.  /usr/local/lib64/libdrmaa.so)
  \item le fully qualified name of the host where is located the PBS/torque
  daemon server
\end{enumerate}

\subsubsection{LsfDRMAAConfig}
is associated to LsfDRMAA ExecutionSystem, which is the interface with
LSF using libdrmaa. This class takes 3 mandatory parameters:
\begin{enumerate}
	\item the path of the drmaa library (eg.  /usr/local/lib64/libdrmaa.so)
  	\item lsf\_envdir the content of the variable ENVDIR of LSF
  	\item lsf\_serverdir the content of the variable SERVERDIR of LSF
\end{enumerate}

\subsubsection{SGEConfig}
is associated to SGE ExecutionSystem, which is the interface with SGE using the shell 
commands wrapping. 
This class takes 2 mandatory parameters:
\begin{enumerate}
  \item root the content of the SGE\_ROOT variable
  \item cell the content of the SGE\_CELL variable
\end{enumerate}

\subsubsection{SYSConfig}
is associated to SYS ExecutionSystem. It is used to launch job without any DRMS.
There is no argument to run a job in ``local''.

\subsection{The Dispatcher}
After having defined all your execution system, you have to specify what system
must be used for a given program. This is the role of the dispatcher.\\
A DefaultDispatcher is provided. 

\subsubsection{The DefaultDispatcher}
associates statically one program to an ExecutionSystem and a queue. 
The DefaultDispatcher takes as argument a dictionary where the
name of the programs are the keys and the values are tuple with 2 arguments.
the first one is an EXECUTION\_SYSTEM\_ALIAS entry and the second a queue name.
There is a joker program name : ``DEFAULT'' to define a system for all programs
which are not listed in the keys.


\section{how to configure}

\subsection{First step: define the execution system you want to use.}
The key is a symbolic name you give to an execution system.\\
The value is an instance of ExecutionConfig.
following an example of EXECUTION\_SYSTEM\_ALIAS using all ExecutionConfig
we provide. 
\begin{verbatim}
from Execution import  *
EXECUTION_SYSTEM_ALIAS = {
          'DRMAA_sge'  : SgeDRMAAConfig( '/usr/local/sge/lib/lx26-amd64/libdrmaa.so' ,
                                         root ='/usr/local/sge',
                                         cell = 'default' ) ,
          'DRMAA_torque': PbsDRMAAConfig('/usr/local/lib64/libdrmaa.so' ,
                                         'marygay.sis.pasteur.fr'),

          'SGE'         : SGEConfig( root = '/usr/local/sge',
                                     cell= 'default' ) ,
          'SYS'        : SYSConfig()  ,
          'LSF'         : LsfDRMAAConfig( '/home/bneron/Sys/lib/libdrmaa.so' ,  
                                                          lsf_envdir = '/home/bneron/Sys/share/lsf/conf' ,
                                                          lsf_serverdir = '/home/bneron/Sys/share/lsf/7.0/linux2.6-glibc2.3-x86_64/etc')
                          }
          }
\end{verbatim}

here a second example to illustrate that you can use the same Execution System with different Config.  

\begin{verbatim}
from Execution import  *
EXECUTION_SYSTEM_ALIAS = {
          'cluster1'  : SgeDRMAAConfig( '/usr/local/sge/lib/lx26-amd64/libdrmaa.so' ,
                                         root ='/usr/local/sge',
                                         cell = 'cluster1' ) ,
          'cluster2' : SgeDRMAAConfig( '/usr/local/sge/lib/lx26-amd64/libdrmaa.so' ,
                                         root ='/usr/local/sge',
                                         cell = 'cluster2' ) ,       
                           }
\end{verbatim}
in this example, the cluster1 and cluster2 have differents features, 
number of nodes, memory \ldots and some jobs must run on cluster1 and the other on cluster2.
 
\subsection{Second step: define which SystemExecution will be used for given program.}
 
After having defined your ExecutionSystem you must specify in what conditions you will use it.
We illustrate here the configuration of the DefaultDispatcher which links a job
name to an ExecutionConfig and a queue.
\begin{verbatim}
from Mobyle.Dispatcher import DefaultDispatcher

DISPATCHER = DefaultDispatcher( {
       'blast2'    : ( EXECUTION_SYSTEM_ALIAS[ 'DRMAA_sge' ] , 'mobyle' ),
       'fastdnaml' : ( EXECUTION_SYSTEM_ALIAS[ 'DRMAA_torque' ] , 'mobyle' ),
       'toppred'   : ( EXECUTION_SYSTEM_ALIAS[ 'DRMAA_torque' ] , 'short' ),
       'dnapars'   : ( EXECUTION_SYSTEM_ALIAS[ 'SGE' ]    , 'long' ),
       'golden'    : ( EXECUTION_SYSTEM_ALIAS[ 'SYS' ]   ,   ''    ),
       'kitch'     : ( EXECUTION_SYSTEM_ALIAS[ 'LSF' ] , 'mobyle' ),
       'DEFAULT'   : ( EXECUTION_SYSTEM_ALIAS[ 'DRMAA_sge' ]    , 'mobyle' )
                             } )
\end{verbatim}
or
\begin{verbatim}
from Mobyle.Dispatcher import DefaultDispatcher

DISPATCHER = DefaultDispatcher( {
       'job1' : ( EXECUTION_SYSTEM_ALIAS[ 'cluster1' ] , 'short' ),
       'job2' : ( EXECUTION_SYSTEM_ALIAS[ 'cluster1' ] , 'long' ),
       'DEFAULT'   : ( EXECUTION_SYSTEM_ALIAS[ 'cluster2' ] , 'mobyle' )
                           } )
\end{verbatim}

Don't be afraid by this configuration once it is done you do not have to
change it very often, and for most of you, you will have only one ExecutionConfig.
For instance if you use SGE and one queue 'mobyle' for all jobs, the
configuration will be:
\begin{verbatim}
EXECUTION_SYSTEM_ALIAS = { 'SGE'  : SGEConfig( root = '/usr/local/sge', cell='default' ) } 
DISPATCHER = DefaultDispatcher( { 'DEFAULT' : ( EXECUTION_SYSTEM_ALIAS[ 'SGE'] , 'mobyle' )
\end{verbatim}

\section{add new execution system}
\label{sec:new_execution_system}
If you have an other execution system not supported by Mobyle, 
you can develop our own ExecutionSystem. This Class must inherits from the abstract ExecutionSystem Class.
The module must contain a class named as the module. Only this class can be used by Mobyle.
Your module must be located in MOBYLEHOME/Src/Execution package.
The new Class must implement 4 methods \_\_init\_\_, \_run , getStatus and kill.

\subsection{\_\_init\_\_}
The init method has one argument which is the ExecutionConfig and is used to do 
all requirements to comunicate with the DRM. 
For instance set some variables in the environment \ldots usually these informations are
contained in the ExecutionConfig. The \_\_init\_\_ method is not necessary if
you don't need any configuration like SYS class.

\subsection{\_run}
This method is the most complex you have to write to implement, and it has several responsabilities.
\begin{itemize}
  \item This method is responsible for submitting the job to the DRMS.
  \item This method must be synchron with the job execution.
  \item As we do not use a server which can keep a record of all jobs, we must store some informations to retrieve a 
given job from an other process (cgi) to get the status of a job or to kill it.
These informations are stored in a '.admin' file located in each job directory.
The \_run method is responsible for setting the value of the execution Sytem used and the key to retrieve this job 
on this Execution system to the an Admin object. The name is always accesible through self.execution\_config\_alias attribute 
and the key is the pid of the job for SYS or the job identifier in SGE\ldots.   
  \item The ADMINDIR directory is a kind of table of all jobs currently in execution in Mobyle. 
It contains a symbolic link toward each job currently running.
The \_run method must make this link when it submits the job to the DRMS and remove it when the job is finished.
  \item Finally, when the job is finished the \_run must map the status job to a Mobyle.Status and return it.
\end{itemize}
There is a Dummy class in the Execution package to help the developer to write his own Execution class.

\subsection{getStatus}
Has one argument, the identifier of the job for this DRMs ( one that you store in .admin file in the \_run method ).
This method queries the DMRS about the status of this job and maps this DRMS status to a Mobyle Status. 
If the job cannot be found in the DRMS the Status ``unknown'' must be returned.

\subsection{kill}
Has one argument, the identifier of the job for this DRMs ( one that you store in .admin file in the \_run method ). 
This method asks the drms to kill the job and return None.

\subsection{ExecutionConfig}
You must implement also the ExecutionConfig which will be associated to this Class. 
The ExecutionConfig Class must be named as the new ExecutionSystem you develop with ``Config'' 
at the end. The module containing this ExecutionConfig
class must be called also as the Class and located in MOBYLEHOME/Local/Config/Execution package.\\

Your ExecutionConfig class must inherit from ExecutionConfig and implement
all requirements needed by your ExecutionSystem you have coded.
At this point you can use your new Execution suitable to your need from the general Config as any 
other provided classes.
   
\section{add new dispatcher}
The Default dispatcher allows to associate one program to one ExecutionSystem
and one queue. This queue is determined statically in the config. If you
need something more dynamic, like compute the quqe based on the email of the
user for instance, you must develop a new Dispatcher. This new dispatcher will
inherit from Dispatcher Class and must implement 2 methods:
\begin{itemize}
  \item getQueue
  \item getExecutionConfig
\end{itemize}

\subsection{getQueue}
Has one argument which is a JobState instance. You can easily access to all
job characteristics (the name of the job, the email of the user\ldots) 
to compute the queue name and return it.   

\subsection{getExecutionConfig}
returns the ExecutionConfig which will used to execute a job.

\section{DRMS requirement if DRMAA is used}

\subsection{python-drmaa}

\subsection{torque}
To work with DRMAA and Mobyle ( to run a synchronous job ) torque must be able
to report the status of a completed jobs. This feature is enabled by setting the keep\_completed attribute on the job execution 
queue or server configuration.

\subsection{LSF}
We need lsf-drmaa from the FedStage DRMAA for LSF project
http://sourceforge.net/projects/lsf-drmaa/
\end{document}