1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-us" lang="en-us"><head>
<style type="text/css">
/* tables */
table.topic {border-collapse:collapse;border: 1px solid black;border-spacing: 0px}
table.topic tr th {text-align:left;font-weight: bold; padding: 2px;}
table.topic tr td {padding: 2px;}
table.topic tr td ul {padding-left:0em; margin-left:1.5em;}
/* contents */
blockquote {border: thin solid black; padding: 1%; margin: 2%;}
dt { font-weight: bold;float: left;display:inline;margin-right: 1em}
dd { display:block; margin-left: 2em}
dl.block dt {float: none;display:block;margin-right: 0em}
dl.signature dd {margin-left: 30em}
</style></head>
<title>ResourceQuotaSpecification</title>
<body bgcolor="#ffffff">
<p>
</p><h1>Specification Resource Quota </h1>
<p>
Roland Dittel<br>
20 November 2006<br>
</p><p>
</p><h2><a name="1_Introduction"> </a> 1 Introduction </h2>
<p>
In large enterprise clusters it is necessary to prevent users from
consuming all available resources. In order to achieve this, N1GE6
supports complex attributes which can be configured on a global, queue
or host layer. This feature is sufficient in certain cases, especially
in small clusters, but has shortcomings and drawbacks for enterprise
usage.
</p><p>Customers have asked for a feature to enhance resource limits
so that they apply to several kinds of resources, several kinds of
resource consumers, to all jobs in the cluster and to combinations of
consumers. In this context, "resources" are any defined complex
attribute (see complex(5)) known by the Grid Engine configuration. For
example this can be slots, arch, mem_total, num_proc, swap_total or any
custom-defined resource like compiler_license.
Resource consumers are (per) users, (per) queues, (per) hosts, (per)
projects, (per) parallel environments. This specification describes a
user interface to define such flexible resource limits.
</p><p>This feature provides a way for administrators to limit the
resources used at a single time by a consumer. However, it is not a way
to define priorities by which user should obtain a resource. Priorities
can be defined by using the Share Tree feature released with N1GE6.
</p><p>
</p><h2><a name="2_Project_Overview"> </a> 2 Project Overview </h2>
<p>
</p><h3><a name="2_1_Project_Aim"> </a> 2.1 Project Aim </h3>
<p>The aim of this project is a solution that allows utilization of
built-in and user-defined resources to be managed in a more flexible
manner. In particular, this is a means to limit resources on a per user
basis and a per project basis. Similarly, resource limitations on the
basis of a user groups and project groups are also required.
</p><p>
The Issues targeted with this project are:
</p><table border="0" cellpadding="0" cellspacing="1" frame="void">
<tbody><tr><th class="twikiFirstCol" align="center" bgcolor="#0080a1"><font color="#ffffff"> Issue</font></th><th align="center" bgcolor="#0080a1"><font color="#ffffff"> Description</font></th></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> <a href="http://gridengine.sunsource.net/issues/show_bug.cgi?id=74">74</a> </font></td><td bgcolor="#ffffff"><font color="#000000"> Support maxujobs on a per host level </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> <a href="http://gridengine.sunsource.net/issues/show_bug.cgi?id=1532">1532</a> </font></td><td bgcolor="#bdbec0"><font color="#000000"> Max jobs per user on a queue basis </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> <a href="http://gridengine.sunsource.net/issues/show_bug.cgi?id=1644">1644</a> </font></td><td bgcolor="#ffffff"><font color="#000000"> Per-user slot limits for limiting PE usage </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> CR 6298406 </font></td><td bgcolor="#bdbec0"><font color="#000000"> Hostgroups should be added as another configuration layer b/w global and host </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> CR 6289250 </font></td><td bgcolor="#ffffff"><font color="#000000"> Request for Job limit per User of Queue </font></td></tr>
</tbody></table>
<p>
</p><h3><a name="2_2_Project_Benefit"> </a> 2.2 Project Benefit </h3>
<p>The expectation is that the management of N1GE cluster resources
will be possible in a much more targeted manner. The enhancement must
make it easy to freely manage limits for arbitrary resources in
relation to existing N1GE objects, such as project/user/host/queue/pe,
without the burden of doing micro-management with countless
projects/users/hosts/queues.
</p><p>
Suggestions for future enhancements are:
</p><ul>
<li> express these resource limits by means of percentages of a wider
context (e.g. (a) memory limit of 4G available for project1 and
project2 (b) up to 70 percent available for project1 and (c) up to 60
percent available for project2) </li>
<li> add a new built-in complex attribute "jobs", that always counts 1 for a job in all resource containers
</li>
<li> define operators which can modify a set of resource limits as a means to allow hierarchical management
</li>
</ul>
<p>
</p><h2><a name="4_Functional_Definition"> </a> 4 Functional Definition </h2>
<h3><a name="4_1_Performance"> </a> 4.1 Performance </h3>
<h3><a name="4_2_Reliability_Availability_Ser"> </a> 4.2 Reliability, Availability, Serviceability (RAS) </h3>
<p>
</p><ul>
<li> Enhancement of Scheduler Profiling
</li>
</ul>
<p>
</p><h3><a name="4_3_Diagnostics"> </a> 4.3 Diagnostics </h3>
<p>
</p><ul>
<li> qquota (new command)
</li>
<li> qstat -j job_id (enhancm.)
</li>
</ul>
<p>
</p><h3><a name="4_4_User_Experience"> </a> 4.4 User Experience </h3>
<p>
</p><h4><a name="4_4_1_Obsolete_Configuration"> </a> 4.4.1 Obsolete Configuration </h4>
<p>
</p><h4><a name="4_4_2_Command_Line_CLI"> </a><a name="4_4_2_Command_Line_CLI_"> </a> 4.4.2 Command Line (CLI) </h4>
<h5><a name="4_4_2_1_CLI_enhancements"> </a> 4.4.2.1 CLI enhancements </h5>
<p>
</p><ul>
<li> qconf switches:
</li>
</ul>
<table border="0" cellpadding="0" cellspacing="1" frame="void">
<tbody><tr><th class="twikiFirstCol" align="center" bgcolor="#0080a1"><font color="#ffffff">switch</font></th><th align="center" bgcolor="#0080a1"><font color="#ffffff">Description</font></th></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -aattr obj_nm attr_nm val obj_id_lst </font></td><td bgcolor="#ffffff"><font color="#000000"> add to a list attribute of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Aaatr obj_nm fname obj_id_lst </font></td><td bgcolor="#bdbec0"><font color="#000000"> add to a list attribute of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -dattr obj_nm attr_nm val obj_id_lst </font></td><td bgcolor="#ffffff"><font color="#000000"> delete from a list attribute of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Dattr obj_nm fname obj_id_lst </font></td><td bgcolor="#bdbec0"><font color="#000000"> delete from a list attribute of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -mattr obj_nm attr_nm val obj_id_lst </font></td><td bgcolor="#ffffff"><font color="#000000"> modify an attribute (or element in a sublist) of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Mattr obj_nm fname obj_id_lst </font></td><td bgcolor="#bdbec0"><font color="#000000"> modify an attribute (or element in a sublist) of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -rattr obj_nm attr_nm val obj_id_lst </font></td><td bgcolor="#ffffff"><font color="#000000"> replace an attribute (or element in a sublist) of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Rattr obj_nm fname obj_id_lst </font></td><td bgcolor="#bdbec0"><font color="#000000"> replace an attribute (or element in a sublist) of an object </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> obj_nm </font></td><td bgcolor="#ffffff"><font color="#000000"> rqs - resource quota set </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> attr_nm </font></td><td bgcolor="#bdbec0"><font color="#000000"> name or enabled or description or limit </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> val </font></td><td bgcolor="#ffffff"><font color="#000000"> new value of attr_nm </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> obj_id_lst </font></td><td bgcolor="#bdbec0"><font color="#000000"> rule set or rule for limit </font></td></tr>
</tbody></table>
<p>
</p><ul>
<li> qstat switches:
</li>
</ul>
<table border="0" cellpadding="0" cellspacing="1" frame="void">
<tbody><tr><th class="twikiFirstCol" align="center" bgcolor="#0080a1"><font color="#ffffff"> switch</font></th><th align="center" bgcolor="#0080a1"><font color="#ffffff"> description</font></th></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -j job_identifier_list </font></td><td bgcolor="#ffffff"><font color="#000000"> show scheduler job information </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -u user_list </font></td><td bgcolor="#bdbec0"><font color="#000000"> view only jobs of this user </font></td></tr>
</tbody></table>
<p>
</p><h5><a name="4_4_2_2_CLI_additions"> </a> 4.4.2.2 CLI additions </h5>
<p>
</p><ul>
<li> qconf switches:
</li>
</ul>
<table border="0" cellpadding="0" cellspacing="1" frame="void">
<tbody><tr><th class="twikiFirstCol" align="center" bgcolor="#0080a1"><font color="#ffffff"> switch</font></th><th align="center" bgcolor="#0080a1"><font color="#ffffff"> Description</font></th></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -arqs [name] </font></td><td bgcolor="#ffffff"><font color="#000000"> add resource quota set(s) </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Arqs fname </font></td><td bgcolor="#bdbec0"><font color="#000000"> add resource quota set(s) from file </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -mrqs [name] </font></td><td bgcolor="#ffffff"><font color="#000000"> modify resource quota set(s) </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -Mrqs fname [name] </font></td><td bgcolor="#bdbec0"><font color="#000000"> modify resource quota set(s) from file </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -srqs [name_list] </font></td><td bgcolor="#ffffff"><font color="#000000"> show resource quota set(s) </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -srqsl </font></td><td bgcolor="#bdbec0"><font color="#000000"> show resource quota set list </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -drqs [name_list] </font></td><td bgcolor="#ffffff"><font color="#000000"> delete resource quota set(s) </font></td></tr>
</tbody></table>
<p>
</p><ul>
<li> qquota switches:
</li>
</ul>
<table border="0" cellpadding="0" cellspacing="1" frame="void">
<tbody><tr><th class="twikiFirstCol" align="center" bgcolor="#0080a1"><font color="#ffffff"> switch</font></th><th align="center" bgcolor="#0080a1"><font color="#ffffff"> description</font></th></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -help </font></td><td bgcolor="#ffffff"><font color="#000000"> print this help </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -h host_list </font></td><td bgcolor="#bdbec0"><font color="#000000"> display only selected host </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -l resource_attributes </font></td><td bgcolor="#ffffff"><font color="#000000"> request the given resources </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -u user_list </font></td><td bgcolor="#bdbec0"><font color="#000000"> display only selected users </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -pe pe_list </font></td><td bgcolor="#ffffff"><font color="#000000"> display only selected parallel environments </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -P project_list </font></td><td bgcolor="#bdbec0"><font color="#000000"> display only selected projects </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#ffffff"><font color="#000000"> -q wc_queue_list </font></td><td bgcolor="#ffffff"><font color="#000000"> display only selected queues </font></td></tr>
<tr><td class="twikiFirstCol" bgcolor="#bdbec0"><font color="#000000"> -xml </font></td><td bgcolor="#bdbec0"><font color="#000000"> display the information in XML-Format </font></td></tr>
</tbody></table>
<p>
</p><h4><a name="4_4_3_Graphical_User_Interface_G"> </a> 4.4.3 Graphical User Interface (GUI) </h4>
<p>
</p><h5><a name="4_4_3_1_Configuration"> </a> 4.4.3.1 Configuration </h5>
<p>
Qmon will be enhanced to allow the configuration of resource resource
quota sets. The configuration will be the same as on CLI with an
editor. </p><p>
</p><h5><a name="4_4_3_2_Diagnose"> </a> 4.4.3.2 Diagnose </h5>
<p>
No Diagnose Support will be provided by qmon
</p><p>
</p><h3><a name="4_5_Manufacturing"> </a> 4.5 Manufacturing </h3>
<p>
</p><ul>
<li> aimk - not affected
</li>
<li> Makefiles - minor changes, new source objects
</li>
</ul>
<p>
</p><h3><a name="4_6_Quality_Assurance"> </a> 4.6 Quality Assurance </h3>
<p>
</p><ul>
<li> New testsuite tests needed
</li>
<li> Modules tests for core module testing
</li>
</ul>
<p>
</p><h3><a name="4_7_Security_Privacy"> </a> 4.7 Security & Privacy </h3>
<p>
Not affected
</p><p>
</p><h3><a name="4_8_Migration_Path"> </a> 4.8 Migration Path </h3>
<ul>
<li> imposes no need for DB update
</li>
<li> imposes no need for update script
</li>
</ul>
<p>
</p><h3><a name="4_9_Documentation"> </a> 4.9 Documentation </h3>
<p>
This Specification is used by Doc writer.
</p><p>
</p><h4><a name="4_9_1_Man_Page_Changes"> </a><a name="4_9_1_Man_Page_Changes_"> </a> 4.9.1 Man Page Changes: </h4>
<p>
</p><ul>
<li> qquota(1) - new man page
</li>
<li> sge_resource_quota(5) - new man page
</li>
<li> qconf(5) - new switches
</li>
</ul>
<p>
</p><h3><a name="4_10_Installation"> </a> 4.10 Installation </h3>
<p> Installation will not not change. For future releases the
installation may change if complex configuration for global/queue/host
becomes obsolete.
</p><p>
At installation time no default rules sets are created.
</p><p>
</p><h3><a name="4_11_Packaging"> </a> 4.11 Packaging </h3>
<p>
Does not change
</p><p>
<p>
</p><h2><a name="5_Component_Descriptions"> </a> 5 Component Descriptions </h2>
<h3><a name="5_1_Component_Resource_Quota_Rul"> </a> 5.1 Component Resource Quota Rules </h3>
<h4><a name="5_1_1_Overview"> </a> 5.1.1 Overview </h4>
<p>According to customers and the filed RFEs it's desired to define a
limit only for specific consumers like users or projects and only for
specific providers like hosts or queues. To achieve this administrators
must be able to define a rule set which consists of the limiting
resource and the limit value, and additionally the consumers or
providers to whom this rule should apply. Because every rule can be
expressed by a tuple of filter specifiers we decided to implement the
rule sets in style of firewall rules.
</p><p>
In practice a rule is defined by:
</p><ul>
<li> <strong>who</strong>
<ul>
<li> users (list of user or usersets/departments)
</li>
<li> projects (list of project)
</li>
</ul>
</li>
<li> <strong>where</strong>
<ul>
<li> parallel_environments (list of pe's)
</li>
<li> hosts (list of host or hostgroups)
</li>
<li> queues (list of cluster queues)
</li>
</ul>
</li>
<li> <strong>what</strong>
<ul>
<li> resource_attribute=max value
</li>
</ul>
</li>
</ul>
<p>The Resource Quota Rules are separate configuration objects and only
used for scheduling decisions. They don't affect the overall cluster
configuration like cluster queues, hosts or projects.
</p><p>
Deliberate use of restrictions in first step of implementation:
</p><ul>
<li> Limits are counted per task as done in the current implementation.
For example if a pe job got 10 slots it will consume 10 licenses.
</li>
<li> Limitation can only be done for fixed and consumable resources, not for load values. <br>
future enhancement: see "Migration Path". That means to have resource
configuration for load values in either global, host for queue level.
</li>
</ul>
<p>
</p><h4><a name="5_1_2_Functionality"> </a> 5.1.2 Functionality </h4>
<p>
</p><h5><a name="Integration_with_current_impleme"> </a> Integration with current implementation </h5>
<p>The Resource Quotas are an addition to the current global, host and
queue instance based scheduling order. The old implementation is still
valid and can be used without the new rules. The rules enhances the old
implementation and adds a new order layer on top of global to define a
more precise limitation. </p><p>
The implications of the layer order on resources are described in
complexes(5) under "Overriding attributes". In general the layers are
AND associated and if one layer denies the job, then the next layer is
ignored. For example, a limit value of "slots=4" can be overwritten in
global, host or queue layer if the layer value is more restrictive, eg,
"slots=2". The exception (see complexes(5)) is for boolean values; for
example "is_linux=true" defined in the tree can not be overwritten to
"is_linux=false" in global host or queue definition. </p><p>
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>resource quotas
|-DENIED->break
|
global
|-DENIED->break
|
host
|-DENIED->break
|
queue
|-DENIED->break
|
OK
</pre>
</td></tr></tbody></table>
<p>
</p><h5><a name="Resource_Reservation"> </a> Resource Reservation </h5>
<p>
Resource Reservation will for Resource Quotas analogue to the current
global/host/queue resource configuration. No changes on client side
necessary.
</p><p>
</p><h4><a name="5_1_3_Interfaces"> </a> 5.1.3 Interfaces </h4>
<p>
</p><h5><a name="Resource_Quota_Set_Syntax"> </a> Resource Quota Set Syntax </h5>
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>ALL: '*'
SEPARATOR: ','
STRING: [^\n]*
QUOTE: '\"'
S_EXPANDER: '{'
E_EXPANDER: '}'
NOT: '!'
BOOL: [tT][rR][uU][eE]
| 1
| [fF][aA][lL][sS][eE]
| 0
NAME: [a-zA-Z][a-zA-Z0-9_-]*
LISTVALUE: ALL | [NOT]STRING
LIST: LISTVALUE [SEPARATOR VALUE]*
NOTSCOPE: LIST | S_EXPANDER LIST E_EXPANDER
SCOPE: ALL | STRING [SEPARATOR STRING]*
RESOURCEPAIR: STRING=STRING
RESOURCE: RESOURCEPAIR [SEPARATOR RESOURCEPAIR]*
rule: "limit" ["name" NAME] ["users" NOTSCOPE] ["projects" SCOPE] ["pes" SCOPE] \
["queues" SCOPE] ["hosts" NOTSCOPE] "to" RESOURCE NL
ruleset_attributes: ("name" NAME NL)
("enabled" BOOL NL)?
("description" QUOTE STRING QUOTE)?
ruleset: "{"
(ruleset_attributes)
(rule)+
"} NL"
rulesets: (ruleset)*
</pre>
</td></tr></tbody></table>
<p>
</p><h5><a name="Resource_Quota_Sets_Format"> </a> Resource Quota Sets Format </h5>
<p>
</p><h6><a name="users"> </a> users </h6>Contains a comma separated
list of UNIX users or ACLs (see access_list(5)). This parameter filters
for jobs by a user in the list or one of the ACLs in the list. Any user
not in the list will not be considered for the resource quota. The
default value is '*' which means any user.
An ACL is differentiated from a UNIX user name by prefixing the ACL
name with an '@' sign. To exclude a user or ACL from the rule the name
can be prefixed with the '!' sign. Defined UNIX user or ACL names need
not be known in the Grid Engine Configuration.
<p>
</p><h6><a name="projects"> </a> projects </h6>Contains a comma
separated list of projects (see project(5)). This parameter filters for
jobs requesting a project of the list. Any project not in the list will
not be considered for the resource quota. If no project filter is
specified all projects and jobs with no requested project matches the
rule. The value '*' means all jobs with requested projects. To exclude
a project from the rule the name can be prefixed with the '!' sign. The
value '!*' means only jobs with no project requested.
<p>
</p><h6><a name="pes"> </a> pes </h6>Contains a comma separated list of
PEs (see sge_pe(5)). This parameter filters for jobs requesting a pe of
the list. Any PE not in the list will not be considered for the
resource quota. If no pe filter is specified all pe and jobs with no
requested pe matches the rule. The value '*' means all jobs with
requested pe. To exclude a pe from the rule the name can be prefixed
with the '!' sign. The value '!*' means only jobs with no pe requested.
<p>
</p><h6><a name="queues"> </a> queues </h6>Contains a comma separated
list of cluster queues (see queue_conf(5)). This parameter filters for
jobs may be scheduled in a queue of the list. Any queue not in the list
will not considered be for the resource quota. The default value is '*'
which means any queue. To exclude a queue from the rule the name can be
prefixed with the '!' sign.
<p>
</p><h6><a name="hosts"> </a> hosts </h6>Contains a comma separated
list of host or hostgroups (see host(5) and hostgroup(5)). This
parameter filters for jobs may be scheduled on a host of the list or a
host contained in the hostgroup. Any host not in the list will not be
considered for the resource quota. The default value is '*' which means
any hosts.
To exclude a host or hostgroup from the rule, the name can be prefixed
with the '!' sign.
<p>
</p><h5><a name="Basic_Configuration"> </a> Basic Configuration </h5>
<p>
</p><h6><a name="Single_Resource_Quota_Rule"> </a> Single Resource Quota Rule </h6>
<p>Resource Quota rules specify the filter criteria that a job must
match and the resulting limit that is taken when a match is found.
</p><p>A rule must always begin with the keyword "limit". The order of
the filter criteria is not important to define and input a rule. After
sending the new rule set to the qmaster the rules will be ordered
automatically to a human readable form.
</p><p>
</p><h6><a name="Scope_Lists"> </a> Scope Lists </h6>
<p>To define a rule for more than one filter scope, it is possible to
group scopes to a list. The defined resource limit counts for all
objects listed in the scope in sum. <br>
For example we have a consumable virtual_free defined as:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>#name shortcut type relop requestable consumable default urgency
#----------------------------------------------------------------------------------------
virtual_free vf MEMORY <= YES YES 1g 0
</pre>
</td></tr></tbody></table>
<p>
In the rule defined below, both users can use together only 5g of virtual_free:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit users roland, andre to virtual_free=5g
</pre>
</td></tr></tbody></table>
<p>
If the administrator wants to limit each of the two users to 5g virtual_free he could define two rules:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit users roland to virtual_free=5g
limit users andre to virtual_free=5g
</pre>
</td></tr></tbody></table>
<p>
This is very cumbersome for large numbers of users or user groups. For
this case a rule can be defined with an expanded list. This would look
like:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit users {roland, andre} to virtual_free=5g
</pre>
</td></tr></tbody></table>
<p>
If the scope contains a usergroup then it gets also expanded and the limit counts also for each member of that group.<br>
For example if a hostgroup @lx_hosts contains host durin and carc both rules are equivalent:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>1)
limit users * hosts durin to virtual_free=10g
limit users * hosts carc to virtual_free=10g
2)
limit users * hosts {@lx_hosts} to virtual_free=10g
</pre>
</td></tr></tbody></table>
<p>
</p><h6><a name="NOT_Operator"> </a> NOT Operator </h6>
<p>Sometimes it is necessary to define a rule for a userset but exclude
some users of that set. This can be defined by using the NOT operator
('!' sign) in front of the user name. A rule so defined will not affect
the excluded user, even if the user is explicitly added to the rule.
</p><p>
For example, user "roland" is also member of usergroup "staff". If a resource quota rule looks like this:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit users @staff,!roland to slots=10
</pre>
</td></tr></tbody></table>
the limit will not be effective for user "roland". Even if the resource quota rule looks like this:
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit users @staff,!roland,roland to slots=10
</pre>
</td></tr></tbody></table>
the rule will not be effective for user "roland"
<p>
</p><h6><a name="Dynamical_Limits"> </a> Dynamical Limits </h6>
<p>Resource Quota rules always define a maximal value of a resource
that can be used. In the most cases these values are static and equal
for all matching filter scopes. If administrators want different rule
limits on different scopes then they have to define multiple rules;
this leads to a duplication of nearly identical rules. With the concept
of dynamical limits this kind of duplication can be avoided.
</p><p>A dynamical limit is a simple algebraic expression used to
derive the rule limit value. To be dynamical the formula can reference
a complex attribute whose value is used for the calculation of the
resulting limit. The limit formula expression syntax is that of a
summation weighted complex values, that is:
</p><p>
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> {w1|$complex1[*w1]}[{+|-}{w2|$complex2[*w2]}[{+|-}...]]
</pre>
</td></tr></tbody></table>
Note, no blanks are allowed in the limit formula.
<p>
The following example clarifies the use of dynamical limits:
Users are allowed to use 5 slots per CPU on all linux hosts.
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> limit hosts {@linux_hosts} to slots=$num_proc*5
</pre>
</td></tr></tbody></table>
<p>The complex attribute num_proc is defined on all hosts and its value
is the processor count on every host. The limit is calculated by the
formula "$num_proc*5" and so is different on some hosts. On a 2 CPU
host users can run 10 slots whereas on a 1 CPU host users only can run
5 slots.
</p><p>
To be able to set the limitation to a well-defined value some prerequisites must be fulfilled
</p><ul>
<li> limit formulas are only possible for INT and DOUBLE.
</li>
<li> The complex must be already defined in the complex list
</li>
<li> The complex must be defined either on global, or queue, or host layer to resolve the value
</li>
<li> The limitation complex must be the same value definition as the
referenced complex value definition (for example slots=INT,
num_proc=INT)
</li>
<li> The resource quota rule must be defined as an expanded list for
the layer the complex is defined. (for example hosts {*} for $num_proc
reference) It's not allowed to reference a complex for a sum of scopes
(for example hosts * for $num_proc).
</li>
</ul>
<p>In principle all INT or DOUBLE kind of complex values could be
referenced but due to time constrains the first implementation allows
only $num_proc in combination with an expanded host list.
</p><p>
</p><h6><a name="Resource_Quota_Rules_and_Resourc"> </a> Resource Quota Rules and Resource Quota Set Interaction </h6>
<p>In practice administrators define some global limits and some limits
that only apply for some resource consumers. These resource quota rules
are equitable. But in some cases it's necessary to define exceptions
for some resource consumers. These resource quota rules are not equal
and dominate some others. As a matter of that fact it is necessary to
allow the definition of a prioritized rule list and a rule list that
apply all of the time. This is done by grouping one or more singe rules
into a number of rule sets.
</p><p>Inside one rule set the rules are ordered and the first rule
found is used. This is analogous to firewall rules and generally
understood by administrators and allows the prioritization of some
rules. A rule set always results in one or none effective resource
quota for a specific request.
</p><p>All of the configured rule sets apply all of the time. This
means if multiple rule sets are defined the most restrictive set is
used and allows to define equitable limits.
</p><p>
The following example clarifies the combination of rules and rule sets. We have a consumable defined as:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>#name shortcut type relop requestable consumable default urgency
#----------------------------------------------------------------------------------------
compiler_lic cl INT <= YES YES 0 0
</pre>
</td></tr></tbody></table>
<p>
The resource quota sets are defined as:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>{
name ruleset1
limit users roland to compiler_lic=3
limit projects * to compiler_lic=2
limit users * to compiler_lic=1
}
{
name ruleset2
limit users * to compiler_lic=20
}
</pre>
</td></tr></tbody></table>
<p>
The first rule set ruleset1 express:
</p><ul>
<li> user roland is allowed to use 3 compiler_lic resources
</li>
<li> any request submitted in a project is allowed to use 2 compiler_lic resources
</li>
<li> the default value for all other users is 1 compiler_lic resource
</li>
</ul>
The second rule set ruleset2 express:
<ul>
<li> all requests together are only allowed to use 20 compiler_lic resources.
</li>
</ul>
<p>Inside ruleset1 the priority is clear defined, user roland will
always get 3 compiler_lic resources even though he matches to "users *"
of the last rule in the rule set and even if he would submit his
request in a project. Also the interaction between ruleset1 and
ruleset2 is clear defined and results in a reject if 20 compiler_lic
resources are already in use, even if user roland does not use all of
his 2 compiler_lic resources.
</p><p>
</p><p>
</p><h5><a name="CLI_Command_Line_Interface"> </a> CLI - Command Line Interface </h5>
<p>
</p><h6><a name="qconf"> </a> qconf </h6>
<p>With qconf it is possible to edit the rule sets in an editor session
like with the most qconf switches. To reduce the amount of data
presented to the administrator its possible to select only one rule set
for editing. </p><p>
It's not possible to edit single rules. Because the rules inside the
rule set are ordered, the meaning of a single rule depends on the
context of all other rules. Therefore it doesn't make sense to edit a
single rule without presenting the context of the rule.
</p><p>
Switch Descriptions:
</p><ul>
<li> <em>-Arqs fname (add RQS configuration)</em>
</li>
</ul>Add the resource quota set (RQS) defined in fname to the Grid
Engine cluster. Returns 0 on success and 1 if rqs is already defined.
Requires root or manager privileges.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ more rule_set.txt
{
name rule_set_2
enabled true
description "rule set 2"
}
$ qconf -Arqs rule_set.txt
rd141302@es-ergb01-01 added "rule_set_2" to resource quota set list
$ qconf -Arqs rule_set.txt
resource quota set "rule_set_2" already exists
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-Mrqs fname [rqs_name] (modify RQS configuration)</em>
</li>
</ul>Same as -mrqs (see below) but instead of invoking an editor to
modify the RQS configuration the file fname is considered to contain a
changed configuration. The name of the rule set in fname must be the
same as rqs_name. If rqs_name is empty all rule sets are overwritten by
the rule sets in fname. Refer to sge_rqs(5) for details on the RQS
configuration format. Returns 0 on success and 1 on error. Requires
root or manager privilege.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ more rule_set.txt
{
name rule_set_3
enabled true
description "rule set 2"
}
$ qconf -Mrqs rule_set.txt rule_set_3
resource quota set "rule_set_3" does not exist
$ qconf -Mrqs rule_set.txt rule_set_4
resource quota set "rule_set_4" does not match rule set definition
$ qconf -Mrqs rule_set.txt
rd141302@es-ergb01-01 modified resource quota set list
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-arqs [name] (add new RQS)</em>
</li>
</ul>Adds a Resource Quota Set (RQS) description under the name
lsr_name to the list of RQSs maintained by Grid Engine (see sge_rqs(5)
for details on the format of a RQS definition). Qconf retrieves a
default RQS configuration and executes vi(1) (or $EDITOR if the EDITOR
environment variable is set) to allow you to customize the RQS
configuration. Upon exit from the editor, the RQS is registered with
sge_qmaster(8). Returns 0 on success and 1 if rqs is already defined.
Requires root/manager privileges.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -arqs
<- {
<- name template
<- enabled true
<- description ""
<- }
-> :q
resource quota set name "template" is not valid
$ qconf -arqs rule_set_1
<- {
<- name rule_set_1
<- enabled true
<- description ""
<- }
-> :wq
rd141302@es-ergb01-01 added "rule_set_1" to resource quota set list
$ qconf -arqs rule_set_1
resource quota set "rule_set_1" already exists
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-mrqs [name] (modify RQS configuration)</em>
</li>
</ul>Retrieves the whole rules or only the specified current
configuration for the resource quota set (RQS), executes an editor
(either vi(1) or the editor indicated by the EDITOR environment
variable) and registers the new configuration with the sge_qmaster(8).
Refer to sge_rqs(5) for details on the RQS configuration format.
Returns 0 on success and 1 on error. Requires root or manager
privilege.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -mrqs rule_set_1
<- {
<- name rule_set_1
<- enabled true
<- description ""
<- }
-> :wq
rd141302@es-ergb01-01 modified "rule_set_1" in resource quota set list
$ qconf -mrqs unknown_set
resource quota set "unknown_set" does not exist
$ qconf -mrqs
<- ...
<- name rule_set_1
<- ...
<- name rule_set_2
<- ...
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-srqs [name_list] (show RQS configuration)</em>
</li>
</ul>
Show the definition of the resource quota set (RQS) specified by the argument.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -srqs
...
name rule_set_1
...
name rule_set_2
...
$ qconf -srqs rule_set_1
...
name rule_set_1
...
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-srqsl (show resource quota sets list)</em>
</li>
</ul>
Show a list of the names of all resource quota sets currently configured.
<p>
</p><ul>
<li> <em>-drqs name_list (delete RQS)</em>
</li>
</ul>Deletes the specified resource quota sets (RQS). Returns 0 on
success and 1 if rqs_name is unknown. Requires root/manager privileges.
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -drqs rule_set_1
rd141302@es-ergb01-01 removed "rule_set_1" from resource quota set list
$ qconf -drqs unknown_rule_set
denied: resource quota set "unknown_rule_set" does not exist
$ qconf -drqs
rd141302@es-ergb01-01 removed resource quota set list
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-aattr obj_nm attr_nm val obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -srqs ruleset_1
{
name ruleset_1
enabled true
limit users @eng to slots=10
limit name arch_rule users @eng to arch=lx24-amd64
}
$ qconf -aattr resource_quota limit slots=20 ruleset_1/1
No modification because "slots" already exists in "limit" of "ruleset_1/1"
$ qconf -aattr resource_quota limit compiler_lic=5 rule_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in rqs list
$ qconf -aattr resource_quota limit arch=sol-sparc64 rule_1/arch_rule
No modification because "arch" already exists in "limit" of "ruleset_1/1"
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-Aaatr obj_nm fname obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ more resource.txt
limit slots=20
$ qconf -Aattr resource_quota resource.txt ruleset_1/1
No modification because "slots" already exists in "limit" of "ruleset_1/1"
$ more resource2.txt
limit compiler_lic=5
$ qconf -Aattr resource_quota resource2.txt ruleset_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in resource_quota list
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-dattr obj_nm attr_nm val obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -dattr resource_quota limit compiler_lic=5 rule_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in rqs list
$ qconf -dattr resource_quota limit compiler_lic=5 rule_1/1
"compiler_lic" does not exist in "limit" of "resource_quota"
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-Dattr obj_nm fname obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ more resource.txt
limit compiler_lic=20
$ qconf -Dattr resource_quota resource.txt rule_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in resource_quota list
$ qconf -Dattr resource_quota resource.txt rule_1/1
"compiler_lic" does not exist in "limit" of "resource_quota"
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-mattr obj_nm attr_nm val obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qconf -mattr resource_quota limit slots=5 rule_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in resource_quota list
$ qconf -mattr resource_quota limit new_resource=5 rule_1/1
Unable to find "new_resource" in "limit" of "resource_quota" - Adding new element.
$ qconf -mattr resource_quota enabled false rule_1
rd141302@es-ergb01-01 modified "ruleset_1" in resource_quota list
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-Mattr obj_nm fname obj_id_lst</em>
</li>
</ul>
See qconf(1)
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ more resource.txt
limit slots=20
$ qconf -Mattr resource_quota resource.txt ruleset_1/1
rd141302@es-ergb01-01 modified "ruleset_1/1" in resource_quota list
$ more resource2.txt
limit new_resource=5
$ qconf -Mattr resource_quota resource2.txt ruleset_1/1
Unable to find "new_resource" in "limit" of "resource_quota" - Adding new element.
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-rattr obj_nm attr_nm val obj_id_lst</em>
</li>
</ul>
See qconf(1)
<p>
</p><ul>
<li> <em>-Rattr obj_nm fname obj_id_lst</em>
</li>
</ul>
See qconf(1)
<p>
</p><h6><a name="qstat"> </a> qstat </h6>
<p>
Switch Descriptions:
</p><p>
</p><ul>
<li> <em>-j job_identifier_list</em>
</li>
</ul>
<p>
Additional Output
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>Example:
cannot run on cluster because exceeds limit in rule_set_1
cannot run on host "bla" because exceeds limit in rule_set_1
cannot run on queue instance "all.q@host" because exceeds limit in rule_set_1
</pre>
</td></tr></tbody></table>
<p>
</p><ul>
<li> <em>-u user_list</em>
</li>
</ul>
<p>
To be consistent with qquota the default value of user_list changes from * (all users) to the calling user.
</p><p>
</p><h6><a name="qquota"> </a> qquota </h6>
<p>
The qquota command is a diagnose tool for the resource resource quotas. The output is a table with the following rows:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>resource quota rule | limit | filter
</pre>
</td></tr></tbody></table>
<p>For each matched rule per rule set a line is printed if the usage
count is not 0 for this rule. If one rule contains more than one
resource attribute then one line is printed per resource attribute.
By default it shows the effective limits for the calling user and for
all other filter criteria like project or pe the wildcard "*" is used
which means not explicit is used.
</p><p>
The output for the limit table is:
</p><ul>
<li> for consumable resources
</li>
</ul>
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> complex=used/limit (for example slots 2/20)
</pre>
</td></tr></tbody></table>
<ul>
<li> for static resources
</li>
</ul>
<table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre> complex=value (for example arch lx24-amd64)
</pre>
</td></tr></tbody></table>
<p>The administrator and the user may define files (analogue to
sge_qstat(5)), which can contain any of the options described below. A
cluster-wide sge_qquota file may be placed under
$SGE_ROOT/$SGE_CELL/common/sge_qquota The user private file is searched
at the location $HOME/.sge_qquota. The home directory request file has
the highest precedence over the cluster global file. Command line can
be used to override the flags contained in the files.
</p><p>
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
Example:
<ul>
<li> All users together should never take more than 20 slots
</li>
<li> All users should maximal take 5 slots on all linux hosts
</li>
<li> Every user is restricted to one slot per linux host, only user
"roland" is restricted to 2 slots and all other slots on hosts are set
to 0
</li>
</ul>
<p>
Rule Set:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>{
name maxujobs
limit users * to slots=20
}
{
name max_linux
limit users * hosts @linux to slots=5
}
{
name max_per_host
limit users roland hosts {@linux} to slots=2
limit users {*} hosts {@linux} to slots=1
limit users * hosts * to slots=0
}
</pre>
</td></tr></tbody></table>
<p>
qstat Output:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qstat
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------
27 0.55500 Sleeper roland r 02/21/2006 15:53:10 all.q@carc 1
29 0.55500 Sleeper roland r 02/21/2006 15:53:10 all.q@carc 1
30 0.55500 Sleeper roland r 02/21/2006 15:53:10 all.q@durin 1
26 0.55500 Sleeper roland r 02/21/2006 15:53:10 all.q@durin 1
28 0.55500 Sleeper user1 r 02/21/2006 15:53:10 all.q@durin 1
</pre>
</td></tr></tbody></table>
<p>
qquota Output:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>$ qquota # as user roland
resource quota rule limit filter
--------------------------------------------------------------------------------
maxujobs/1 slots=5/20 -
max_linux/1 slots=5/5 hosts @linux
max_per_host/1 slots=2/2 users roland hosts durin
max_per_host/1 slots=2/2 users roland hosts carc
$ qquota -h durin # as user roland
resource quota limit filter
--------------------------------------------------------------------------------
maxujobs/1 slots=5/20 -
max_linux/1 slots=5/5 hosts @linux
max_per_host/1 slots=2/2 users roland hosts durin
$ qquota -u user1
resource quota limit filter
--------------------------------------------------------------------------------
maxujobs/1 slots=5/20 -
max_linux/1 slots=5/5 hosts @linux
max_per_host/1 slots=1/2 users user1 hosts durin
$ qquota -u *
resource quota limit filter
--------------------------------------------------------------------------------
maxujobs/1 slots=5/20 -
max_linux/1 slots=5/5 hosts @linux
max_per_host/1 slots=2/2 users roland hosts carc
max_per_host/1 slots=2/2 users roland hosts durin
max_per_host/1 slots=1/2 users user1 hosts durin
</pre>
</td></tr></tbody></table>
</td></tr></tbody></table>
<p>
qquota XML Schema:
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre><?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified">
<xsd:element name="qquota_result">
<xsd:sequence>
<xsd:element name="qquota_rule" type="QQuotaRuleType" minOccurs="0" maxOccurs="unbounded"/>
</xsd:sequence>
</xsd:element>
<xsd:complexType name="QQuotaRuleType">
<xsd:sequence>
<xsd:element name="user" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="xuser" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="project" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="xproject" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="pe" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="xpe" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="queue" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="xqueue" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="host" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="xhost" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
<xsd:element name="limit" type="ResourceLimitType" minOccurs="0" maxOccurs="unbounded"/>
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required"/>
</xsd:complexType>
<xsd:complexType name="ResourceLimitType">
<xsd:attribute name="resource" type="xsd:string" use="required"/>
<xsd:attribute name="limit" type="xsd:string" use="required"/>
<xsd:attribute name="value" type="xsd:string" use="optional"/>
</xsd:complexType>
</xsd:schema>
</pre>
</td></tr></tbody></table>
<p>
</p><h5><a name="Internal_data_structures"> </a><a name="Internal_data_structures_"> </a> Internal data structures: </h5>
<p>
</p><h6><a name="Additional_Cull_Lists"> </a> Additional Cull Lists </h6>
<p>
All lists are used by qmaster and scheduler
</p><p>
File sge_resource_quotaL.h
</p><table bgcolor="#ffffff" border="1" cellpadding="3" cellspacing="1" width="100%"><tbody><tr><td>
<pre>#ifndef __SGE_RESOURCE_QUOTAL_H
#define __SGE_RESOURCE_QUOTAL_H
#include "sge_boundaries.h"
#include "cull.h"
#ifdef __cplusplus
extern "C" {
#endif
/* *INDENT-OFF* */
/* Resource Quota Set */
enum {
RQS_name = RQS_LOWERBOUND,
RQS_description,
RQS_enabled,
RQS_rule
};
LISTDEF(RQS_Type)
JGDI_ROOT_OBJ(ResourceQuotaSet, SGE_RQS_LIST, ADD | MODIFY | DELETE | GET | GET_LIST)
JGDI_EVENT_OBJ(ADD(sgeE_RQS_ADD) | MODIFY(sgeE_RQS_MOD) | DELETE(sgeE_RQS_DEL) | GET_LIST(sgeE_RQS_LIST))
SGE_STRING(RQS_name, CULL_PRIMARY_KEY | CULL_HASH | CULL_UNIQUE | CULL_SPOOL)
SGE_STRING(RQS_description, CULL_DEFAULT | CULL_SPOOL)
SGE_BOOL(RQS_enabled, CULL_DEFAULT | CULL_SPOOL)
SGE_LIST(RQS_rule, RQR_Type, CULL_DEFAULT | CULL_SPOOL)
LISTEND
NAMEDEF(RQSN)
NAME("RQS_name")
NAME("RQS_description")
NAME("RQS_enabled")
NAME("RQS_rule")
NAMEEND
#define RQSS sizeof(RQSN)/sizeof(char*)
/* Resource Quota Rule */
enum {
RQR_name = RQR_LOWERBOUND,
RQR_filter_users,
RQR_filter_projects,
RQR_filter_pes,
RQR_filter_queues,
RQR_filter_hosts,
RQR_limit,
RQR_level
};
LISTDEF(RQR_Type)
JGDI_OBJ(ResourceQuotaRule)
SGE_STRING(RQR_name, CULL_PRIMARY_KEY | CULL_HASH | CULL_UNIQUE | CULL_SPOOL)
SGE_OBJECT(RQR_filter_users, RQRF_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_OBJECT(RQR_filter_projects, RQRF_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_OBJECT(RQR_filter_pes, RQRF_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_OBJECT(RQR_filter_queues, RQRF_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_OBJECT(RQR_filter_hosts, RQRF_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_LIST(RQR_limit, RQRL_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_ULONG(RQR_level, CULL_DEFAULT | CULL_JGDI_RO)
LISTEND
NAMEDEF(RQRN)
NAME("RQR_name")
NAME("RQR_filter_users")
NAME("RQR_filter_projects")
NAME("RQR_filter_pes")
NAME("RQR_filter_queues")
NAME("RQR_filter_hosts")
NAME("RQR_limit")
NAME("RQR_level")
NAMEEND
#define RQRS sizeof(RQRN)/sizeof(char*)
enum {
FILTER_USERS = 0,
FILTER_PROJECTS,
FILTER_PES,
FILTER_QUEUES,
FILTER_HOSTS
};
enum {
RQR_ALL = 0,
RQR_GLOBAL,
RQR_CQUEUE,
RQR_HOST,
RQR_QUEUEI
};
/* Resource Quota Rule Filter */
enum {
RQRF_expand = RQRF_LOWERBOUND,
RQRF_scope,
RQRF_xscope
};
LISTDEF(RQRF_Type)
JGDI_OBJ(ResourceQuotaRuleFilter)
SGE_BOOL(RQRF_expand, CULL_DEFAULT | CULL_SPOOL)
SGE_LIST(RQRF_scope, ST_Type, CULL_DEFAULT | CULL_SPOOL)
SGE_LIST(RQRF_xscope, ST_Type, CULL_DEFAULT | CULL_SPOOL)
LISTEND
NAMEDEF(RQRFN)
NAME("RQRF_expand")
NAME("RQRF_scope")
NAME("RQRF_xscope")
NAMEEND
#define RQRFS sizeof(RQRFN)/sizeof(char*)
/* Resource Quota Rule Limit */
enum {
RQRL_name = RQRL_LOWERBOUND,
RQRL_value,
RQRL_type,
RQRL_dvalue,
RQRL_usage,
RQRL_dynamic
};
LISTDEF(RQRL_Type)
JGDI_OBJ(ResourceQuotaRuleLimit)
SGE_STRING(RQRL_name, CULL_PRIMARY_KEY | CULL_HASH | CULL_UNIQUE | CULL_SPOOL)
SGE_STRING(RQRL_value, CULL_DEFAULT | CULL_SPOOL)
SGE_ULONG(RQRL_type, CULL_DEFAULT | CULL_SPOOL | CULL_JGDI_RO)
SGE_DOUBLE(RQRL_dvalue, CULL_DEFAULT | CULL_SPOOL | CULL_JGDI_RO)
SGE_LIST(RQRL_usage, RUE_Type, CULL_DEFAULT | CULL_JGDI_RO)
SGE_BOOL(RQRL_dynamic, CULL_DEFAULT | CULL_JGDI_RO)
LISTEND
NAMEDEF(RQRLN)
NAME("RQRL_name")
NAME("RQRL_value")
NAME("RQRL_type")
NAME("RQRL_dvalue")
NAME("RQRL_usage")
NAME("RQRL_dynamic")
NAMEEND
#define RQRLS sizeof(RQRLN)/sizeof(char*)
/* *INDENT-ON* */
#ifdef __cplusplus
}
#endif
#endif /* __SGE_RESOURCE_QUOTAL_H */
</pre>
</td></tr></tbody></table>
<p>
</p><h6><a name="Additional_GDI_requests"> </a> Additional GDI requests </h6>
<p>
</p><ul>
<li> SGE_GDI_ADD(RQS, resource_quota)
</li>
</ul>This request allows for adding a new resource quota set. It
contains the complete rule set configuration and is used for
implementing the qconf option '-arqs' and '-Arqs'.
<p>
</p><ul>
<li> SGE_GDI_MOD(RQS, resource_quota)
</li>
</ul>This request allows for changing the complete resource quota set
configuration. It contains a full rule set configuration and is used
for implementing qconf option '-mrqs' and '-Mrqs'.
<p>
</p><ul>
<li> SGE_GDI_DEL(RQS, resource_quota)
</li>
</ul>This request allows for removing a complete resource quota set
configuration. It contains only the name of the resource quota to be
removed and is used for implementing the qconf option '-drqs'.
<p>
</p><ul>
<li> SGE_GDI_GET(RQS,where,what)
</li>
</ul>This request allows for retrieving resource quota sets. CULL
'where' expressions can be used for selecting particular rule sets,
CULL 'what' expressions can be used for selecting particular rule set
fields. The SGE_GDI_GET request is used for implementing the qconf
option '-srqs'.
<p>
</p><ul>
<li> SGE_GDI_MOD(LISR, resource_quota, fields)
</li>
<li> SGE_GDI_MOD(LISR, resource_quota, fields) + SGE_GDI_SET()
</li>
</ul>These requests are a SGE_GDI_MOD(LISR, resource_quota) variation
and allow for replacing the selected fields within a resource quota.
Field selection is done by means of an incomplete resource quota set
configuration structure. The requests are used for implementing qconf
options '-rattr' and '-Rattr'.
<p>
</p><ul>
<li> SGE_GDI_MOD(LISR, resource_quota, fields) + SGE_GDI_APPEND(rule_identifiers, list_elements)
</li>
</ul>This request allows for adding one or more list elements regarding
to one or more rule identifiers to each of the selected list fields
within a resource quota set configuration. Field selections are done by
means of an incomplete rule set configuration structure. The
rule_identifiers of each tuple below each selected rule set field are
used to define which rule should be modified. All list elements
belonging to each tuple are added. Already existing list elements are
silently overwritten, also if the selected rule configuration is not a
list field this silently overwrites the current setting.The request is
for implementing the qconf option '-aatrr' and '-Aattr'.
<p>
</p><ul>
<li> SGE_GDI_MOD(LISR, resource_quota, fields) + SGE_GDI_CHANGE(rule_identifiers, list_elements)
</li>
</ul>This request allows for replacing one or more list elements
regarding to one or more rule identifiers to each of the selected list
fields within a resource quota set configuration. Field selections are
done by means of an incomplete rule set configuration structure. The
request is for implementing the qconf option -mattr' and '-Mattr'
<p>
</p><ul>
<li> SGE_GDI_MOD(RQS, resource_quota, fields) + SGE_GDI_REMOVE(rule_identifiers, list_elements)
</li>
</ul>This request allows for removing one or more list elements
regarding to one or more rule identifiers to each of the selected list
fields within a resource quota set configuration. Field selections are
done by means of an incomplete rule set configuration structure. The
request is for implementing the qconf option -dattr' and '-dattr'
<p>
</p><h6><a name="Additional_Event_Client_requests"> </a> Additional Event Client requests </h6>
<p>
</p><ul>
<li> sgeE_RQS_LIST
</li>
</ul>This event is sent once directly after event client registration
to initialize the resource quota set list and contains the complete
list of all resource quota sets with all configuration.
<p>
</p><ul>
<li> sgeE_RQS_ADD(resource_quota)
</li>
</ul>This event is sent each time when a new resource quota set
configuration has been created. It contains the full resource quota set
configuration, but no usage information.
<p>
</p><ul>
<li> sgeE_RQS_DEL(resource_quota)
</li>
</ul>This event is sent each time when an existing resource quota set
configuration is removed and contains only the name of the resource
quota to be removed.
<p>
</p><ul>
<li> sgeE_RQS_MOD(resource_quota)
</li>
</ul>This event is sent each time when an existing resource quota set
configuration changes. It contains the full resource quota set
configuration.
<p>
</p><ul>
<li> sgeE_RQS_ADD(resource_quota, rule_identifier, usage)
</li>
<li> sgeE_RQS_MOD(resource_quota, rule_identifier, usage)
</li>
<li> sgeE_RQS_DEL(resource_quota, rule_identifier, usage)
</li>
</ul>These events are send each time when a usage object was added,
modified or deleted. The resource_quota and rule_identifier contains
only the name of the object to be edited. The usage object is the
object to be modified.
<p>
</p><h6><a name="Qmaster_additions"> </a><a name="Qmaster_additions_"> </a> Qmaster additions: </h6>
<ul>
<li> add cull rule set definition (Internal data structures)
</li>
<li> spooling code for the rule sets (mainly for classic spooling)
</li>
<li> update resource usage in all rules
</li>
</ul>
<p>
</p><h6><a name="Scheduler_additions"> </a><a name="Scheduler_additions_"> </a> Scheduler additions: </h6>
<ul>
<li> Create Resource Reservation Structure
<ul>
<li> prepare_resource_schedules()
</li>
</ul>
</li>
<li> Scheduling matching code
<ul>
<li> sge_sequential_assignment()
</li>
<li> sge_select_parallel_environment()
</li>
</ul>
</li>
<li> Debit Code
<ul>
<li> debit_scheduled_job()
</li>
</ul>
</li>
</ul>
<p>
</p><h6><a name="lib_additions"> </a><a name="lib_additions_"> </a> lib additions: </h6>
<ul>
<li> add code for book keeping of resource usage
</li>
</ul>
<p>
</p><h6><a name="Book_keeping_of_usage"> </a><a name="Book_keeping_of_usage_"> </a> Book keeping of usage: </h6>
<ul>
<li> started jobs
</li>
<li> finished/deleted/running jobs
</li>
<li> (suspended jobs)
</li>
<li> object modify (queue host)
</li>
<li> object add/delete (queue host)
</li>
</ul>
<hr>
</body></html>
|
