1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" dir="ltr" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="keywords" content="EvaluationExpressionSupport">
</head>
<body class="ns-0 ltr">
<div id="globalWrapper">
<div id="column-content">
<div id="content">
<a name="top" id="top"></a>
<h1 class="firstHeading">EvaluationExpressionSupport</h1>
<div id="bodyContent">
<h3 id="siteSub">Date 2007/01/22</h3>
<table id="toc" class="toc" summary="Contents"><tbody><tr><td><div id="toctitle"><h2>Contents</h2> </div>
<ul>
<li class="toclevel-1"><a href="#Evaluation_Expression_Support"><span class="tocnumber">1</span> <span class="toctext">Evaluation Expression Support</span></a>
<ul>
<li class="toclevel-2"><a href="#Introduction"><span class="tocnumber">1.1</span> <span class="toctext">Introduction</span></a>
<ul>
<li class="toclevel-3"><a href="#Project_Overview"><span class="tocnumber">1.1.1</span> <span class="toctext">Project Overview</span></a></li>
<li class="toclevel-3"><a href="#Project_Aim"><span class="tocnumber">1.1.2</span> <span class="toctext">Project Aim</span></a></li>
<li class="toclevel-3"><a href="#Wildcard_Matching_Defined_in_sge_types.281.29"><span class="tocnumber">1.1.3</span> <span class="toctext">Wildcard Matching Defined in sge_types(1)</span></a></li>
<li class="toclevel-3"><a href="#Complex_String_Attributes"><span class="tocnumber">1.1.4</span> <span class="toctext">Complex String Attributes</span></a></li>
<li class="toclevel-3"><a href="#RESTRING"><span class="tocnumber">1.1.5</span> <span class="toctext">RESTRING</span></a></li>
<li class="toclevel-3"><a href="#Hostname_Expression_Support"><span class="tocnumber">1.1.6</span> <span class="toctext">Hostname Expression Support</span></a></li>
<li class="toclevel-3"><a href="#Support_for_New_qsub_-q_.3Cexpr.3E.40.3Cexpr.3E"><span class="tocnumber">1.1.7</span> <span class="toctext">Support for New qsub -q <expr>@<expr></span></a></li>
<li class="toclevel-3"><a href="#Allowed_Character_Restriction"><span class="tocnumber">1.1.8</span> <span class="toctext">Allowed Character Restriction</span></a></li>
</ul>
</li>
<li class="toclevel-2"><a href="#Project_Benefit"><span class="tocnumber">1.2</span> <span class="toctext">Project Benefit</span></a></li>
</ul>
</li>
<li class="toclevel-1"><a href="#System_Enhancement"><span class="tocnumber">2</span> <span class="toctext">System Enhancement</span></a>
<ul>
<li class="toclevel-2"><a href="#Enhancement_Functions"><span class="tocnumber">2.1</span> <span class="toctext">Enhancement Functions</span></a>
<ul>
<li class="toclevel-3"><a href="#qsub.2Cqsh.2Cqrsh.2Cqalter.2Cqresub.2Cqlogin"><span class="tocnumber">2.1.1</span> <span class="toctext">qsub,qsh,qrsh,qalter,qresub,qlogin</span></a></li>
<li class="toclevel-3"><a href="#qconf"><span class="tocnumber">2.1.2</span> <span class="toctext">qconf</span></a></li>
<li class="toclevel-3"><a href="#qselect"><span class="tocnumber">2.1.3</span> <span class="toctext">qselect</span></a></li>
<li class="toclevel-3"><a href="#qstat"><span class="tocnumber">2.1.4</span> <span class="toctext">qstat</span></a></li>
<li class="toclevel-3"><a href="#qacct"><span class="tocnumber">2.1.5</span> <span class="toctext">qacct</span></a></li>
<li class="toclevel-3"><a href="#qmod"><span class="tocnumber">2.1.6</span> <span class="toctext">qmod</span></a></li>
</ul>
</li>
</ul>
</li>
<li class="toclevel-1"><a href="#Documentation_Changes"><span class="tocnumber">3</span> <span class="toctext">Documentation Changes</span></a>
<ul>
<li class="toclevel-2"><a href="#Man_Pages"><span class="tocnumber">3.1</span> <span class="toctext">Man Pages</span></a>
<ul>
<li class="toclevel-3"><a href="#submit.281.29"><span class="tocnumber">3.1.1</span> <span class="toctext">submit(1)</span></a></li>
<li class="toclevel-3"><a href="#qalter"><span class="tocnumber">3.1.2</span> <span class="toctext">qalter</span></a></li>
<li class="toclevel-3"><a href="#sge_types.281.29"><span class="tocnumber">3.1.3</span> <span class="toctext">sge_types(1)</span></a></li>
<li class="toclevel-3"><a href="#complex.285.29"><span class="tocnumber">3.1.4</span> <span class="toctext">complex(5)</span></a></li>
<li class="toclevel-3"><a href="#qstat.281.29"><span class="tocnumber">3.1.5</span> <span class="toctext">qstat(1)</span></a></li>
<li class="toclevel-3"><a href="#qacct.281.29"><span class="tocnumber">3.1.6</span> <span class="toctext">qacct(1)</span></a></li>
<li class="toclevel-3"><a href="#qselect.281.29"><span class="tocnumber">3.1.7</span> <span class="toctext">qselect(1)</span></a></li>
</ul>
</li>
</ul>
</li>
</ul>
</td></tr></tbody></table><script type="text/javascript"> if (window.showTocToggle) { var tocShowText = "show"; var tocHideText = "hide"; showTocToggle(); } </script>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=1" title="Edit section: Evaluation Expression Support">edit</a>]</div><a name="Evaluation_Expression_Support"></a><h1> Evaluation Expression Support </h1>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=2" title="Edit section: Introduction">edit</a>]</div><a name="Introduction"></a><h2> Introduction </h2>
<p>N1GE 6.0 supports a RESTRING in a complex configuration allowing to
specify an OR delimited "|" list of patterns for limiting the jobs.
</p><p>A support for more flexible RESTRING expressions (see
complex(5)) is possible. A more flexible syntax for pattern matching
will be introduced in future releases.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=3" title="Edit section: Project Overview">edit</a>]</div><a name="Project_Overview"></a><h3> Project Overview </h3>
<p>SGE 6.0 introduced a new complex type RESTRING that allows logical
OR connected regular expressions for string resources. SGE 6.1 will
introduce a more flexible syntax for pattern matching. Specifically,
the NOT operator ("don't schedule my job to a 'linux' host")
</p><p>Examples:
</p>
<pre> qsub -l arch="!linux" ... don't schedule my job to a 'linux'
qsub -l arch="!linux|hp" ... would work while you couldn't express "don't
schedule my job to a 'linux' but on 'hp' host"
qsub -l arch="!(linux|hp)" ... since you need to express it as not this nor this
</pre>
<p>More or less, this also makes it automatically convenient to support a logical AND.
</p><p>Examples:
</p>
<pre> qsub -l arch="!(lx*|sol*)&*64*" ... schedule my job on a 64-bit OS, but not on Solaris or Linux
qsub -l myos="(rh_3*|suse_[89]*)&!rh_3.1" ... schedule my job on a host which has the "myos" attribute rh_3*,
suse_8*, or suse_9*, but do not schedule it on a rh_3.1 host
</pre>
<p>The regular boolean expression definition can be found in the sge_types(1) man page.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=4" title="Edit section: Project Aim">edit</a>]</div><a name="Project_Aim"></a><h3> Project Aim </h3>
<p>The new feature implementation of the regular boolean expression for
the wildcard expressions or patterns should be implemented in the
maintrunk.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=5" title="Edit section: Wildcard Matching Defined in sge_types(1)">edit</a>]</div><a name="Wildcard_Matching_Defined_in_sge_types.281.29"></a><h3> Wildcard Matching Defined in sge_types(1) </h3>
<p>Any wildcard matching for the following
</p>
<ul><li> wc_project_name [optional]
</li><li> wc_user_list_name [optional]
</li><li> wc_queue
</li><li> wc_cqueue
</li><li> wc_qdoamin
</li><li> wc_qinstance
</li><li> wc_host
</li><li> wc_hostgroup
</li><li> wc_pe [optional]
</li></ul>
<p>will support the boolean expression matching function sge_eval_expression().
</p><p>The optional types will be implemented if some real requirement is identified.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=6" title="Edit section: Complex String Attributes">edit</a>]</div><a name="Complex_String_Attributes"></a><h3> Complex String Attributes </h3>
<p>All the complex string attributes (CSTRING, STRING, RESTRING, HOST)
will support boolean wildcard expressions. This wildcard matching will
be implemented by the sge_eval_expression() function.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=7" title="Edit section: RESTRING">edit</a>]</div><a name="RESTRING"></a><h3> RESTRING </h3>
<p>RESTRING becomes obsolete. RESTRING can be kept for interface
compatibility but STRING and RESTIRNG will use the same matching code.
This behavior would be documented. The RESTRING would become deprecated
[complex(5)] interface and would disappear with the next major (7.0)
release.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=8" title="Edit section: Hostname Expression Support">edit</a>]</div><a name="Hostname_Expression_Support"></a><h3> Hostname Expression Support </h3>
<p>Since there is already implemented support of several matching
routines for hostnames (IGNORE_FQDN={true|false}, DEFAULT_DOMAIN), the
original implementation behavior would not be changed.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=9" title="Edit section: Support for New qsub -q &lt;expr&gt;@&lt;expr&gt;">edit</a>]</div><a name="Support_for_New_qsub_-q_.3Cexpr.3E.40.3Cexpr.3E"></a><h3> Support for New qsub -q <expr>@<expr> </h3>
<p>Since the queue attribute -l q=XXX and the host attribute -l h=YYY
support the new expression/pattern matching the "-q" also should
support the new matching.
</p><p>qsub -q <expr>@<expr>
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=10" title="Edit section: Allowed Character Restriction">edit</a>]</div><a name="Allowed_Character_Restriction"></a><h3> Allowed Character Restriction </h3>
<p>A clearer definition of allowed characters for an attribute, in it's
definition and in the resource request, should be defined. Currently,
everything is allowed, including the (for qacct) dangerous ":".
</p><p>Allowed characters could be a broad range a-z,A-Z,0-9,_,-,#,=,<,>,.,,(comma)
</p><p>All characters could be allowed at all positions of the attributes or hostnames. Only valid hostnames will be accepted.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=11" title="Edit section: Project Benefit">edit</a>]</div><a name="Project_Benefit"></a><h2> Project Benefit </h2>
<p>The adherence to accepted industry standards of the syntax for
wildcard expression matching definitions will provide a wider
acceptance among the customer base.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=12" title="Edit section: System Enhancement">edit</a>]</div><a name="System_Enhancement"></a><h1> System Enhancement </h1>
<p>The enhancement is targeting whole range of commands in a uniform and consistent way. We add this as a standard feature.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=13" title="Edit section: Enhancement Functions">edit</a>]</div><a name="Enhancement_Functions"></a><h2> Enhancement Functions </h2>
<p>The new expression syntax will be supported in different commands.
The new meaning of some characters requires certain verifications to be
done and could introduce some restrictions on attribute values.
Reserved characters change the behavior. Characters in range
[,],?,*,&,|,! are accounted as an expression reserved words with
syntactical meanings in patterns and they are not allowed in the
compared values.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=14" title="Edit section: qsub,qsh,qrsh,qalter,qresub,qlogin">edit</a>]</div><a name="qsub.2Cqsh.2Cqrsh.2Cqalter.2Cqresub.2Cqlogin"></a><h3> qsub,qsh,qrsh,qalter,qresub,qlogin </h3>
<p>An overview of command line option changes:
</p>
<pre> -l complex=<expression> ... support for all string type complexes for wildcard expressions.
-q <wc_queue_list> ... support for hostnames, queuenames, groupnames as the wildcard expression
-masterq <wc_queue_list>
-pe parallel_environment (optional - see above)
</pre>
<p>Parallel environment wildcard match
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=15" title="Edit section: qconf">edit</a>]</div><a name="qconf"></a><h3> qconf </h3>
<pre> -cq <wc_queue_list> .... cleans queue
-sq <wc_queue> .... displays one or multiple cluster queues or queue instances
-purge <wc_queue> .... purges a queue instances
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=16" title="Edit section: qselect">edit</a>]</div><a name="qselect"></a><h3> qselect </h3>
<pre> -q <wc_queue> .... wildcard expression queue list to be included in the output
-l complex=<expression> ... support for all string type complexes for wildcard expressions
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=17" title="Edit section: qstat">edit</a>]</div><a name="qstat"></a><h3>qstat</h3>
<pre> -q <wc_queue_list>
-q <wc_queue_list> -g c
-l complex=<expression>
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=18" title="Edit section: qacct">edit</a>]</div><a name="qacct"></a><h3>qacct</h3>
<pre> -q <wc_queue>
-l complex=<expression>
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=19" title="Edit section: qmod">edit</a>]</div><a name="qmod"></a><h3>qmod</h3>
<pre> -e <wc_queue_list> .... enable queue expression
-d <wc_queue_list> .... disable queue expression
-c <wc_queue_list>
-cq <wc_queue_list> ... clear error status queue expression
-rq <wc_queue_list>
</pre>
<p>Reschedule queue expression
</p>
<pre> -sq <wc_queue_list> .... suspend queue expression
-usq <wc_queue_list> .... 'unsuspend' queue expression
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=20" title="Edit section: Documentation Changes">edit</a>]</div><a name="Documentation_Changes"></a><h1>Documentation Changes</h1>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=21" title="Edit section: Man Pages">edit</a>]</div><a name="Man_Pages"></a><h2>Man Pages</h2>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=22" title="Edit section: submit(1)">edit</a>]</div><a name="submit.281.29"></a><h3> submit(1) </h3>
<p>There is a reference to the new sge_types(1) man page describing the
wildcard value. On several places an expression is mentioned instead of
a pattern.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=23" title="Edit section: qalter">edit</a>]</div><a name="qalter"></a><h3> qalter </h3>
<p>There is a reference to the new sge_types(1) man page decribing the wildcard value.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=24" title="Edit section: sge_types(1)">edit</a>]</div><a name="sge_types.281.29"></a><h3> sge_types(1) </h3>
<p>The general syntax for a wildcard expression would look as:
</p>
<ul><li> expression = ["!"] ["("] valExp [")"] [ AND_OR expression ]
</li><li> valExp = pattern | expression
</li><li> AND_OR = "&" | "|"
</li></ul>
<pre> "!" ... not operator - negate the following pattern or expression
"&" ... and operator - logical and with the following expression
"|" ... or operator - logical or with the following expression
"(" ... open bracket - begins the priority evaluated expression. The close bracket is required
")" ... close bracket - ends the priority evaluated expression.
"pattern" ... see the \fIpattern\fP definition
</pre>
<p>The wildcard expression itself must be put into '"' to ensure that the clients get the expression.
</p><p>The 'pattern' can be found in the sge_types(1) man page. It is a
pattern expression supported by the standard C-lib fnmatch() function.
</p><p>Examples:
</p>
<pre> "!(lx*|sol*)&*64*" ... not begin with lx or sol but contain 64
"(rh_3*|suse_[89]*)&!rh_3.1" ... attribute rh_3*, suse_8*, or suse_9* but not rh_3.1
</pre>
<p>WARNING: Reserved characters change the behavior! Characters in
range [,],?,*,&,|,! are accounted as an expression reserved words
with syntactical meanings in patterns and they are not allowed in the
compared values.
</p><p>This means that it is not allowed to set the complex for particular queue to the value containing this reserved characters.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=25" title="Edit section: complex(5)">edit</a>]</div><a name="complex.285.29"></a><h3> complex(5) </h3>
<p>The general syntax for a wildcard complex expressions is now referenced by sge_types(1) to avoid duplicate declaration
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=26" title="Edit section: qstat(1)">edit</a>]</div><a name="qstat.281.29"></a><h3> qstat(1) </h3>
<p>On several places an expression is mentioned instead of a pattern
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=27" title="Edit section: qacct(1)">edit</a>]</div><a name="qacct.281.29"></a><h3> qacct(1) </h3>
<p>The general syntax for a wildcard complex expressions is now in
sge_types(1), so there is a simplification of some command description.
</p>
<div class="editsection" style="float: right; margin-left: 5px;">[<a href="http://wiki.gridengine.info/wiki/index.php?title=EvaluationExpressionSupport&action=edit&section=28" title="Edit section: qselect(1)">edit</a>]</div><a name="qselect.281.29"></a><h3> qselect(1) </h3>
<p>In one place there is mentioned an expression instead of a pattern
</p>
<!-- Saved in parser cache with key gridwikidb:pcache:idhash:1565-0!1!0!0!!en!2 and timestamp 20070122134034 -->
<div class="printfooter">
Retrieved from "<a href="http://wiki.gridengine.info/wiki/index.php/EvaluationExpressionSupport">http://wiki.gridengine.info/wiki/index.php/EvaluationExpressionSupport</a>"</div>
<!-- end content -->
</div>
</div>
</div>
</div>
</body></html>
|
