1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=iso-8859-1">
<TITLE></TITLE>
<META NAME="GENERATOR" CONTENT="StarOffice/5.2 (Solaris Sparc)">
<META NAME="AUTHOR" CONTENT=" ">
<META NAME="CREATED" CONTENT="20010611;14564800">
<META NAME="CHANGEDBY" CONTENT=" ">
<META NAME="CHANGED" CONTENT="20010716;16471800">
</HEAD>
<BODY>
<H1><FONT SIZE=6 STYLE="font-size: 28pt">DRMAA C binding library for Grid Engine</FONT></H1>
<H1>Overview</H1>
<P>The purpose of this library is to provide an implementation of the DRMAA C binding
standard based on Grid Engine DRM system. The best way to describe DRMAA is to cite
the DRMAA-WG Charter:</P>
<UL>
<P>"Develop an API specification for the submission and control of jobs to one or more Distributed Resource
Management (DRM) systems. The scope of this specification is all the high level functionality which is
necessary for an application to consign a job to a DRM system including common operations on jobs like
termination or suspension.</P>
<P>The objective is to facilitate the direct interfacing of applications to today's DRM systems by
application's builders, portal builders, and Independent Software Vendors (ISVs)."</P>
</UL>
<P>For more details on the
<A HREF="http://www.gridforum.org">Global Grid Forum</A>
standards body working group refer to the page of the
<A HREF="http://www.drmaa.org">GGF DRMAA working group</A>
.
</P>
<H1>Documentation</H1>
<P>The following DRMAA related documentation is recommended
<A HREF="http://www.drmaa.org/docs/ggf-rec-drmaa-1_0.pdf">DRMAA 1.0 language independent specification</A>,
<A HREF="https://forge.gridforum.org/tracker/?group_id=90&atid=608&aid=214">DRMAA C language binding specification</A>,
<A HREF="http://gridengine.sunsource.net/unbranded-source/browse/%7Echeckout%7E/gridengine/source/libs/japi/jobstates.html?content-type=text/html">Grid Engine/DRMAA job state mapping table</A>
and
<A HREF="http://gridengine.sunsource.net/unbranded-source/browse/~checkout~/gridengine/doc/htmlman/manuals.html?content-type=text/html">HTML version of Grid Engine DRMAA C binding man pages (only those in section 3 with drmaa_* prefixes)</A>
.</P>
<H1>Availability</H1>
<P>For the current 5.3 release DRMAA is not and will not be available. For 6.0 it is planned to deliver
everything needed to integrate applications via DRMAA with Grid Engine including a DRMAA library that
can be linked dynamically. At this stage DRMAA is available as a source snapshot from the "maintrunk"
of the Grid Engine. To get this source snapshot you must use the cvs option "-r VMAIN_STABLE_3_TAG" when
you check out the sources as described under <A HREF="http://gridengine.sunsource.net/servlets/ProjectSource">Download Source</A>.</P>
<P>In this source snapshot the following Grid Engine binary architectures are known to be working:</P>
<UL>
<LI><P>solaris (Sparc Solaris 2.6, 7, 8, 9 32-bit)</P>
><P>solaris64 (Sparc Solaris 7, 8, 9 64-bit</P>
<LI><P>solaris86 (x86 Solaris 8)</P>
><P>glinux (x86 Linux, kernel 2.4, glibc >= 2.2)</P>
</UL>
<P>There are three build targets that are related to DRMAA:</P>
<UL>
<LI><P>libdrmaa.a - a DRMAA library for statically linking</P>
<LI><P>libjapi.a - a JAPI library for statically linking</P>
<LI><P>test_drmaa - a test DRMAA application used for QA</P>
</UL>
<H1>Building</H1>
<P>The steps described in the
<A HREF="http://gridengine.sunsource.net/unbranded-source/browse/~checkout~/gridengine/source/README.BUILD">Grid Engine build page</A>
are the same that also must be followed for building the DRMAA build targets. In case of Grid Engine binary architectures where the
port is already done the command</P>
<UL>
aimk
</UL>
<P>compiles all regular Grid Engine binaries plus DRMAA build targets. To enforce compilation
of DRMAA build targets for not yet ported binary architectures
<UL>
aimk -only-mt
</UL>
<P>can be used. To compile all Grid Engine binaries except the DRMAA build targets</P>
<UL>
aimk -no-mt
</UL>
<P>can be used.</P>
<P>Upon successful build the DRMAA targets are found in the "*_MT"
directories. Build targets in this directory are compiled in a way allowing them to be
used in multi-threaded applications. Due to the JAPI service thread (see implementation of
<A HREF="http://gridengine.sunsource.net/unbranded-source/browse/%7Echeckout%7E/gridengine/source/libs/japi/japi.html?content-type=text/html">JAPI library</A>
) this is necessary for major parts of the code that implements DRMAA. For the same reason a
good amount of Grid Engine source code has been reworked to make it reentrant (where possible)
or at least thread safe. As result of this effort MT safe functions are marked with comments
such as
</P>
<UL>
MT-NOTE: sge_gdi() is MT safe
</UL>
<P>it is important to understand that with all these comments major assumptions were
made. The current major assumptions are</P>
<UL>
<LI><P>code was not compiled with aimk option -DCOMMCOMPRESS</P>
<LI><P>code was not compiled with aimk option -DCRYPTO</P>
<LI><P>code was not compiled with aimk option -DKERBEROS</P>
<LI><P>code was not compiled with aimk option -SECURE</P>
<LI><P>if the code was compiled with aimk option -SECURE then
only non-secure may be used</P>
<LI><P>neither AFS nor DCE/KERBEROS security may be used</P>
<LI><P>system is not used in I18N mode</P>
</UL>
<P>if these assumptions are not met DRMAA library can not be expected to be working
properly and will crash most probably! To overcome those constraints it is necessary
firstly to link Grid Engine against the corresponding MT safe libraries (zlib,
openssl, ...). Secondly those parts of the Grid Engine source code that makes use of
those libraries must be reviewed and changed were necessary.</P>
<H1>Example application</H1>
<P>The C test program
<A HREF="http://gridengine.sunsource.net/source/browse/gridengine/source/libs/japi/example.c">example.c</A>
is a good example of an application that uses the DRMAA C binding interface.
It illustrates submission of both single and bulk remote jobs. After submission drmaa_synchronize() call
is used to synchronize the remote jobs execution. The call returns after all the jobs have finished
executing. Finally, drmaa_wait() call is used to retrieve and print out the remote jobs' execution
information.</P>
<P>A full path for the remote command is passed as the first argument to the test program.
That value is directly used as "drmaa_remote_command" job template attribute. The C binding example uses
value "5" as a first argument to the job template vector attribute "drmaa_v_argv". Passing "/bin/sleep" as
a first argument to the test program will for example cause 32 sleep jobs to be run that sleep for "5"
seconds each before finishing execution. Note that we expect to find "/bin/sleep" command on all of the
remote nodes.</P>
<P>To build this example run
<UL>
aimk -no-mt example
</UL>
This is necessary because "example" is not part of the standard build procedure.</P>
<H1>Bugs</H1>
You can file bugs to Grid Engine Issuezilla "drmaa" subcomponent. Before doing so please have a
look at the TODO list below and watch out for already submitted "drmaa" bugs. If you're not sure
whether you found a bug sending mail to
<A HREF="http://gridengine.sunsource.net/project/gridengine/maillist.html">dev@gridengine.sunsource.net Mailinglist</A>
is recommended.
<H1>Notes on implementation</H1>
<P>The Grid Engine implementation of DRMAA bases on Grid Engine
<A HREF="http://gridengine.sunsource.net/unbranded-source/browse/%7Echeckout%7E/gridengine/source/libs/japi/japi.html?content-type=text/html">Job API library (JAPI)</A>
. As a result DRMAA library inherits MT safety from JAPI library.</P>
<P>Interfaces operations of Grid Engine JAPI library and DRMAA library have lots in common.
The big difference amongst them is that JAPI allows any characteristic of the Grid Engine job
that is submitted through JAPI be influenced, whereas DRMAA allows only those characteristics be influenced that
are interfaced by DRMAA. Note that this ostensible drawback of DRMAA library is not as meaningful as it seems:
Firstly DRMAA job template attributes "drmaa_job_category" and "drmaa_native_specification" will allow nearly all
of the qsub(1) job characteristics be influenced. Secondly by building an integration upon DRMAA library you
don't need to change your application code each time when a new Grid Engine version is released. When using JAPI
library interface you must be prepared to change your application code in this case. This is because JAPI directly
exports Grid Engine internal data structures that usually are subject of change with new releases.</P>
<H1>Grid Engine DRMAA library QA</H1>
<P>The DRMAA application "test_drmaa" is used by Grid Engine testsuite QA tool to ensure quality of DRMAA
library. It covers a growing number of tests. There are so-called standalone tests that can
be accomplished autonomously by the test_drmaa application. To run all these tests standalone "test_drmaa"
can be started with the "ALL_AUTOMATED" option plus some option arguments. To run all
DRMAA related tests so far implemented choose "api/drmaa" in testsuite and have testsuite do the work for you.</P>
<P ALIGN=CENTER STYLE="text-decoration: none">Copyright 2003 Sun Microsystems, Inc. All rights reserved.</P>
</BODY>
</HTML>
|
