1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<!-- $Id: develop.htm,v 1.12 2004/11/24 21:37:02 jfontain Exp $ -->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Author" content="Jean-Luc Fontaine">
<meta name="Description" content="reference manual on writing modules for moodss">
<title>moodss (Modular Object Oriented Dynamic SpreadSheet) modules development</title>
</head>
<body bgcolor="#FFFFFF" style="font-family: helvetica, verdana, arial">
<center><h2>Developing modules for moodss (Modular Object Oriented Dynamic SpreadSheet)</h2></center>
<h3>Contents:</h3>
<ul>
<li><a href="#architecture">1. Architecture</a>
<li><a href="#source">2. Source</a><ul>
<li><a href="#namespace">2.1. Namespace</a>
<li><a href="#configuration">2.2. Configuration</a>
<li><a href="#initialization">2.3. Initialization</a>
<li><a href="#termination">2.4. Termination</a>
<li><a href="#variabledata">2.5. Variable data</a>
<li><a href="#rows">2.6. Row numbering</a>
<li><a href="#internationalization">2.7. Internationalization</a>
</ul>
<li><a href="#installation">3. Installation</a>
<li><a href="#displaying">4. Displaying messages</a>
<li><a href="#errors">5. Error handling</a>
<li><a href="#daemon">6. Daemon specifics</a>
<li><a href="#perl">7. Perl</a>
<li><a href="#python">8. Python</a>
<li><a href="#threadsevents">9. Threads and events programming</a>
<li><a href="#utilities">10. Utilities</a><ul>
<li><a href="#hash">10.1. Hash library</a>
<li><a href="#linetask">10.2. Non-blocking channel library</a>
</ul>
<li><a href="#tips">11. Tips and tricks</a><ul>
<li><a href="#singlerow">11.1 Single row tables</a>
<li><a href="#void">11.2. Void values as numbers</a>
</ul>
</ul>
<p><b>Note</b>: all examples are drawn from the <i>random</i> Tcl and <i>Random</i> Perl sample modules.
<h3><a name="architecture"></a>1. Architecture</h3>
<p>The moodss and moomps applications are composed of the core software and one or several modules. Modules are implemented as Tcl packages and thus usually comprise a Tcl, Perl or Python source file and a pkgIndex.tcl file as required by the Tcl package implementation.
<p>All the module code and data are kept in a separate namespace. The module data is stored is a single array including some configuration data used when the module is loaded by the core, and variable data (displayed in the application table and possibly graphical viewers).
<br>If a module is synchronous, it must start updating its data when requested by the core. If a module is asynchronous, its data may be updated at any time. The synchronous or asynchronous nature is specified in the configuration data for the module.
<h3><a name="source"></a>2. Source</h3>
<p>A module is a package, it must have a name and a version.
<br><i><b>Note</b>: version number management is important as the module major version number is taken into account when storing data cells history in a database (can be done from both the moodss GUI application and the moomps daemon). Modules developers must insure that no major changes occur between minor releases of the module. In other words, if there is a change in the module data structure, such as adding a new column, changing the order of columns, changing column data, such as label or type (message and anchor can be altered without problems), then a new major version number <b>must</b> be used, to preserve database integrity (a new major version number results in the new data to be stored with new, thus different identifiers).</i>
<ul>
<li>
<b>Tcl</b>: at the beginning of the <i>random.tcl</i> file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package provide random 1.1
</pre></td><tr></table>
This line simply states that this is the <i>1.1</i> version of the <i>random</i> package. Please note that the package name must also be used as the module namespace name (see below).<br><br>
<li>
<b>Perl</b>:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package Random;
...
BEGIN {
...
our $VERSION = 1.1;
}
</pre></td><tr></table>
Note that traditionally, Perl package names must be capitalized, and the package name must match the corresponding file name (not counting the <i>.pm</i> extension). The <i>VERSION</i> variable must be defined.<br><br>
<li>
<b>Python</b>: at the beginning of the <i>randpy.py</i> file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
__version__ = 1.1
</pre></td><tr></table>
The package name is the name of the Python module, defined by the module file name (<i>randpy</i> for <i>randpy.py</i>).
</ul>
<p>Module names can be any combination of the following characters (minus the characters not allowed by the specific module language, of course):<ul>
<li><tt>space character</tt>
<li><tt>alphanumeric (letters and digits)</tt>
<li><tt>_@%&*()=+:.-</tt>
</ul>
<h4><a name="namespace"></a>2.1. Namespace</h4>
<p>All module procedures and data are kept in a specific namespace bearing the module name.
<ul>
<li>
<b>Tcl</b>: from the <i>random.tcl</i> file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
namespace eval random {
array set data {
...
}
proc update {} {
...
}
}
</pre></td><tr></table>
<li>
<b>Perl</b>: implicit as a module has its own namespace:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
our %data;
our @data;
...
sub update() {
...
</pre></td><tr></table>
<li>
<b>Python</b>: implicit as a module has its own namespace (symbol table):
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
form = {}
data = []
...
def update():
...
</pre></td><tr></table>
</ul>
<p>The update function is not needed when the module is asynchronous.
<p><i><b>Note</b>: at this time, it is required for Perl and Python modules, as they cannot be asynchronous.</i>
<h4><a name="configuration"></a>2.2. Configuration</h4>
<p>The module configuration defines the data table column headers, help information, ... This data never changes during the lifetime of the application.
<ul>
<li>
<b>Tcl</b>: all the module configuration data is stored as array members of the array named <i>data</i> within the module namespace. For example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
namespace eval random {
array set data {
updates 0
0,label name 0,type ascii 0,message {user name}
1,label cpu 1,type real 1,message {cpu usage in percent}
2,label disk 2,type integer 2,message {disk usage in megabytes}
3,label command 3,type dictionary 3,message {most time consuming command} 3,anchor left
pollTimes {10 5 20 30 60 120 300}
sort {1 decreasing}
indexColumns {0 3}
helpText {This is a simple demonstration module ...}
}
...
}
</pre></td><tr></table>
<li>
<b>Perl</b>: all the module configuration data is stored in a hash named <i>data</i>.
<br>Columns data is kept in an array of hashes. The array index is the column number starting from 0.
<br>The poll times and index columns are kept in arrays, while the sort data is stored in a hash.
<br>For example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
our %data;
$data{updates} = 0;
$data{columns}[0] = {
label => 'name', type => 'ascii', message => 'user name'
};
$data{columns}[1] = {
label => 'cpu', type => 'real', message => 'cpu usage in percent'
};
$data{columns}[2] = {
label => 'disk', type => 'integer', message => 'disk usage in megabytes'
};
$data{columns}[3] = {
label => 'command', type => 'dictionary', message => 'command name',
anchor => 'left'
};
$data{pollTimes} = [10, 5, 20, 30, 60, 120, 300];
$data{sort} = {1 => 'decreasing'};
$data{indexColumns} = [0, 3];
$data{helpText} = 'This is a simple demonstration module ...';
...
</pre></td><tr></table>
<li>
<b>Python</b>: all the module static configuration data is stored in a dictionary named <i>form</i>, while the dynamic data is kept in a dictionary named <i>data</i>.
<br>Columns data is stored in a list (indexed by column number) of dictionaries.
<br>The poll times and index columns are kept in lists, while the sort data is stored in a dictionary.
<br>For example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
form = {}
form['updates'] = 0
form['columns'] = [
{'label': 'name', 'type': 'ascii', 'message': 'user name'},
{'label': 'cpu', 'type': 'real', 'message': 'cpu usage in percent'},
{'label': 'disk', 'type': 'integer', 'message': 'disk usage in megabytes'},
{'label': 'memory', 'type': 'integer', 'message': 'memory usage in kilobytes'},
{'label': 'command', 'type': 'dictionary', 'message': 'command name', 'anchor': 'left'}
]
form['pollTimes'] = [10, 5, 20, 30, 60, 120, 300]
form['sort'] = {1: 'decreasing'}
form['indexColumns'] = [0, 4]
form['helpText'] = 'This is a simple demonstration module ...'
...
</pre></td><tr></table>
</ul>
<p>The <i>updates</i> member is a counter used to keep track of the number of times that the module data was updated, and is also used by the core to detect when module data display should be updated (see <a href="#variabledata">variable data</a> for more information).
<p>The <i>label</i> members (<i>$data(<b>n</b>,label)</i> in Tcl, <i>$data{columns}[<b>n</b>]{label}</i> in Perl, <i>form['columns'][<b>n</b>]['label']</i> in Python, with <i><b>n</b></i> the column number), define the text to be displayed as column titles. There must be as many <i>label</i> members as they are columns. The titles must contain no <i>?</i> character.
<p>The <i>type</i> members (<i>$data(<b>n</b>,type)</i> in Tcl, <i>$data{columns}[<b>n</b>]{type}</i> in Perl, <i>form['columns'][<b>n</b>]['type']</i> in Python) define the type of the corresponding column data. Valid types are simply those that the Tcl <i>lsort</i> command can handle: <i>ascii</i>, <i>dictionary</i>, <i>integer</i> and <i>real</i>, plus the <i>clock</i> type, which accepts any format that the Tcl <i>clock format</i> command can handle (see the <a href="#trace">trace</a> module for an example). There must be as many <i>type</i> members as they are columns.
<p>The <i>message</i> members (<i>$data(<b>n</b>,message)</i> in Tcl, <i>$data{columns}[<b>n</b>]{message}</i> in Perl, <i>form['columns'][<b>n</b>]['message']</i> in Python) define the text of the help message to be displayed in a floating yellow window (widget tip, balloon, also see <a href="#architecture">User Interface</a>) as the user moves the mouse pointer over column titles. It can be composed of only a few words or multiple formatted lines. There must be as many <i>message</i> members as they are columns.
<p>The <i>anchor</i> member (<i>$data(<b>n</b>,anchor)</i> in Tcl, <i>$data{columns}[<b>n</b>]{anchor}</i> in Perl, <i>form['columns'][<b>n</b>]['anchor']</i> in Python) is optional. Column data is either centered by default, tucked to the left or right side of the column. Valid values are <i>center</i>, <i>left</i> or <i>right</i>.
<p>Note that column numbers start at <b>0</b>. There must be <b>no</b> hole in the column numbers sequence.
<p>The <i>pollTimes</i> member is a list of valid poll times (in seconds) for the module. The list is not ordered, as its first element represents the default poll time value to be used when the moodss application starts. This value may be overridden by a command line argument. The smallest value in the list is used by the core as the lowest possible poll time and checked against when the user enters a new value through the poll time dialog box. The list must not be empty.
<br>Note that the list is also used by moodss as a set of possible choices in the dialog box used to set the new poll time. The user may still directly input any value as long as it is greater than or equal to the minimum value.
<p>If the module is asynchronous (data can be updated at any time and not in response to update procedure invocations (no polling required)), the pollTimes member must be a single negative integer value representing the preferred time interval for viewers that require one (only graphs at this point). For example, if you wish graph viewers to have a display interval of 10 seconds, use:
<ul>
<li>
<b>Tcl</b>:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
namespace eval random {
array set data {
...
pollTimes -10
...
}
...
}
</pre></td><tr></table>
<li>
<b>Perl</b>: asynchronous modules not yet available.
<li>
<b>Python</b>: asynchronous modules not yet available.
</ul>
<p>In this case, the graph viewers time range (knowing that they feature 100 time points) would be 1000 seconds. I guess that the value that you should specify as the pollTimes member should be the expected average update interval for your asynchronous data. Note that the graphical viewers x axis always display properly labeled absolute time ticks in any case.
<p>When several asynchronous modules are loaded with no synchronous modules, the interval used for all relevant viewers is the average (in absolute value) of all module intervals. For example, if you load 2 asynchronous modules, one with a pollTimes member of -10 and the other of -20, then a 15 seconds interval value is retained. Note that the interval can be forced through the --poll-time command line argument.
<p>If at least one synchronous module is loaded concurrently with any number of asynchronous modules, the actual application poll time (the one that can be set with the then available poll time dialog box) is used.
<p>The <i>indices</i> member is an optional list that specifies the table columns that should be displayed. If not specified, all the table columns are visible.
<p>The <i>sort</i> entry is optional. It defines the index of the column which should be initially used as a reference for sorting the data table rows, and in which order (<i>increasing</i> or <i>decreasing</i>) the rows should be sorted. The column index for sorting works like the <i>-index</i> Tcl <i>lsort</i> command option, that is rows are sorted so that that specific column appears sorted in the specified order. The specified column must be visible (see <i>indices</i> member documentation above).
<p>The <i>indexColumns</i> list specifies the columns required to uniquely identify a row in the table. In database talk, it represents the table key. To maintain backward compatibility, it is optional and defaults to 0, the leftmost column. The index columns are used when creating data viewer elements: their label is built by concatenating the key value for the cell row with the cell column title. The key value is the concatenation of the index column values for the cell. When specified, all the columns in the list must be visible (see <i>indices</i> member documentation above).
<p>The <i>helpText</i> member specifies a text of any length, to be displayed when the user requests help information on the current module from within the help menu. The text can be HTML formatted or plain. HTML formatted help requires the <HTML> and <BODY> tags to be present, while tables and frames are not supported (and many other tags: stick to formatted text at the moment). The core will render HTML formatted or plain text in the module help window according to the help text contents.
<p>The <i>views</i> member is optional. If specified, it defines one or more views to be used in place of the default view. One table will be displayed per view. For each view, 1 member must be defined: <i>indices</i>, the <i>sort</i> member being optional (syntax and usage are identical to the default table members). A <i>swap</i> optional member may be used if the table data is to be displayed with columns and rows swapped, which is generally the case when the data table has 1 or 2 rows, or a fixed number of rows (see <a href="#memstats">memstats</a> module for an example). The <i>swap</i> value is a boolean and must be either <b>0</b> or <b>1</b>.
<ul>
<li>
<b>Tcl</b>: each view is a list of array members, suitable as an argument to an "array set" command:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
namespace eval random {
array set data {
...
views {
{indices {0 1 3 4} sort {1 decreasing} swap 1}
{indices {0 2 4} sort {2 decreasing}}
}
...
}
...
}
</pre></td><tr></table>
<li>
<b>Perl</b>: each view is a hash:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
$data{views} = [
{indices => [0, 1, 3, 4], sort => {1 => 'decreasing'}, swap => 1},
{indices => [0, 2, 4], sort => {2 => 'decreasing'}}
];
</pre></td><tr></table>
<li>
<b>Python</b>: each view is a dictionary:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
form['views'] = [
{'indices': [0, 1, 3, 4], 'sort': {1: 'decreasing'}, 'swap': 1},
{'indices': [0, 2, 4], 'sort': {2: 'decreasing'}}
]
</pre></td><tr></table>
</ul>
<p>The <i>switches</i> member is optional. A switch is a single letter or a string with the - or + sign as header. The boolean value (0 or 1) specifies whether the switch takes an argument. If the switches member exists, an appropriate initialize procedure must be provided by the module (see <a href="#initialization">Initialization</a>). The core will take care of parsing the command line and reject any invalid switch / value combination for the module. The switches value may not be changed in the initialize procedure.
<p><b>Note</b>: if you wish to pass a sensitive password as an option to the module, use a special option name (<i>-passwd</i> or <i>-password</i> or <i>--passwd</i> or <i>--password</i>) so that the core knows when not to display or store readable characters, for example when entering a password in a dialog box, or storing password value in the moomps daemon data cells history database.
<p>For example, a recent <i>random</i> module configuration is as follows:
<ul>
<li>
<b>Tcl</b>: switches are a list of switch / boolean pairs:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
array set data {
updates 0
0,label name 0,type ascii 0,message {user name}
1,label cpu 1,type real 1,message {cpu usage in percent}
2,label disk 2,type integer 2,message {disk usage in megabytes}
3,label memory 3,type integer 3,message {memory usage in kilobytes}
4,label command 4,type dictionary 4,message {command name}
pollTimes {10 5 20 30 60 120 300}
sort {1 decreasing}
indexColumns {0 4}
helpText {...}
views {
{indices {0 1 3 4} sort {1 decreasing}}
{indices {0 2 4} sort {2 decreasing}}
}
switches {-a 0 --asynchronous 0}
}
</pre></td><tr></table>
<li>
<b>Perl</b>: switches are stored in a hash:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
our %data;
$data{updates} = 0;
$data{columns}[0] = {label => 'name', type => 'ascii', message => 'user name'};
$data{columns}[1] = {label => 'cpu', type => 'real', message => 'cpu usage in percent'};
$data{columns}[2] = {label => 'disk', type => 'integer', message => 'disk usage in megabytes'};
$data{columns}[3] = {label => 'memory', type => 'integer', message => 'memory usage in kilobytes'};
$data{columns}[4] = {label => 'command', type => 'dictionary', message => 'command name'};
$data{pollTimes} = [10, 5, 20, 30, 60, 120, 300];
$data{sort} = {1 => 'decreasing'};
$data{indexColumns} = [0, 4];
$data{helpText} = '...';
$data{views} = [
{indices => [0, 1, 3, 4], sort => {1 => 'decreasing'}},
{indices => [0, 2, 4], sort => {2 => 'decreasing'}}
];
$data{switches} = {'-a' => 0, '--asynchronous' => 0};
</pre></td><tr></table>
<li>
<b>Python</b>: switches are stored in a dictionary:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
form = {}
form['updates'] = 0
form['columns'] = [
{'label': 'name', 'type': 'ascii', 'message': 'user name'},
{'label': 'cpu', 'type': 'real', 'message': 'cpu usage in percent'},
{'label': 'disk', 'type': 'integer', 'message': 'disk usage in megabytes'},
{'label': 'memory', 'type': 'integer', 'message': 'memory usage in kilobytes'},
{'label': 'command', 'type': 'dictionary', 'message': 'command name'}
]
form['pollTimes'] = [10, 5, 20, 30, 60, 120, 300]
form['sort'] = {1: 'decreasing'}
form['indexColumns'] = [0, 4]
form['helpText'] = '...'
form['views'] = [
{'indices': [0, 1, 3, 4], 'sort': {1: 'decreasing'}},
{'indices': [0, 2, 4], 'sort': {2: 'decreasing'}}
]
form['switches'] = {'-a': 0, '--asynchronous': 0}
</pre></td><tr></table>
</ul>
In this case, data is presented in 2 tables: one for the CPU and memory usage, the other for disk usage.
<p>The <i>identifier</i> member is optional. It is string that uniquely identifies this module. If set, it is displayed by the core in the initial data tables title area and data cell labels in viewers. This feature can be used for example in modules that gather data from a remote host: in such a case, the identifier could be set to <i>dataType(hostName)</i> (the ps module uses <i>ps(host)</i> string). The allowed character set for the identifier is identical to the allowed set for a module name.
<br>Note that the <i>identifier</i> member is usually set in the module <i>initialize</i> procedure, as it usually depends on the module options.
<p>The <i>resizableColumns</i> member is optional. It is a boolean value (<i>0</i> or <i>1</i>) which specifies whether displayed data table(s) columns can be manually resized by the user with the mouse. Not available on a per view basis. If not present or set to <i>0</i>, all the module data columns are automatically sized according to their content.
<p>The <i>persistent</i> member is optional. It is a boolean value (<i>0</i> or <i>1</i>) that tells whether row numbers are identically mapped from data row keys across module instances. Most modules should have that capability as viewers, thresholds, database history, ... rely on the internal row/column pair to identify a cell: see <a href="#rows">row numbering</a> section for detailed information. If absent, the module is considered non-persistent.
<p>The <i>64Bits</i> member is optional. It is a boolean value (<i>0</i> or <i>1</i>) which specifies whether row numbers are unsigned 64 bit integers (which requires a Tcl 8.4 or above core version). If absent or false, row numbers as unsigned 32 bit integers are assumed.
<h4><a name="initialization"></a>2.3. Initialization</h4>
<p>The <i>initialize</i> procedure, if it exists, is invoked by the core before any update occurs (when the update procedure is invoked if the module is synchronous).
<p>In order for a module to accept command line arguments, the initialize procedure must exist (more information below). Otherwise, if the module can take no arguments, it stays optional. In that case, it is rather redundant with in-line module code (outside of any module function or procedure). When the module takes no arguments, so does the initialize procedure if it exists.
<p>The initialize procedure is mandatory when the module supports command line arguments, and in such a case takes option values as arguments.
<p>Let us use the following command line as example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
$ moodss random --asynchronous --other-option value -x 1234
</pre></td><tr></table>
<ul>
<li>
<b>Tcl</b>: The initialize procedure takes an array name as sole argument. The array contains the switched options values, indexed by switch. The options array will contain:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
options(--asynchronous) =
options(--other-option) = value
options(-x) = 1234
</pre></td><tr></table>
Note that the <i>--asynchronous</i> member value is empty as that switch takes no argument. For the above example, switches would have been defined as:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
switches {-a 0 --asynchronous 0 --other-option 1 -x 1}
</pre></td><tr></table>
<li>
<b>Perl</b>: The initialize procedure takes a hash as sole argument. The hash contains the switched options values, indexed by switch. The hash will contain:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
$options{--asynchronous} = 1
$options{--other-option} = value
$options{-x} = 1234
</pre></td><tr></table>
Note that the <i>--asynchronous</i> member value is filled with a boolean even though that switch takes no argument. For the above example, switches would have been defined as:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
$data{switches} =
{'-a' => 0, '--asynchronous' => 0, '--other-option' => 1, '-x' => 1};
</pre></td><tr></table>
<li>
<b>Python</b>: The initialize function takes a dictionary as sole argument. The dictionary contains the switched options values, indexed by switch. The dictionary will contain:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
options['--asynchronous'] = 1
options['--other-option'] = value
options['-x'] = 1234
</pre></td><tr></table>
Note that the <i>--asynchronous</i> member value is filled with a boolean even though that switch takes no argument. For the above example, switches would have been defined as:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
form['switches'] =\
{'-a': 0, '--asynchronous': 0, '--other-option': 1, '-x': 1}
</pre></td><tr></table>
</ul>
<p>In all cases, data members other than <i>updates</i> and <i>switches</i> can be set or updated in the initialize procedure, and successfully taken into account by the core.
<p>Example for a module that take arguments, where a module identifier is generated according to the <i>-i</i> or <i>--identify</i> command line switches:
<ul>
<li>
<b>Tcl</b>: the options array is accessed using the <i>upvar</i> command technique (used when passing arguments (such as arrays) by name):
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
proc initialize {optionsName} {
upvar $optionsName options
variable data
if {[info exists options(-i)] || [info exists options(--identify)]} {
# generate a random module identifier:
set data(identifier) "random [expr {int(rand() * 100)}]"
}
}
</pre></td><tr></table>
<li>
<b>Perl</b>: the options are passed as a hash with the switch as index, where options that take no arguments have a boolean as value:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
sub initialize(%) {
my %option = @_;
if ($option{'-i'} || $option{'--identify'}) {
# generate a random module identifier:
my $identifier = int(rand(100));
$data{identifier} = "random $identifier";
}
}
</pre></td><tr></table>
<li>
<b>Python</b>: the options are passed as a dictionary with the switch as index, where options that take no arguments have a boolean as value:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
def initialize(options):
global form
identify = 0
try: identify = options['--identify']
except:
try: identify = options['-i']
except: pass
if identify:
form['identifier'] = 'randpy ' + str(randint(0, 100));
</pre></td><tr></table>
</ul>
<p>The core waits for the initialize procedure to be completed before initializing the next module. For modules likely to initialize slowly and/or susceptible to initialization failure, it is advised to allow the user interface to be updated in the meantime:
<ul>
<li>
<b>Tcl</b>: using the <i>vwait</i> command (technique used in all the remote capable modules shipped with moodss):
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
proc initialize {optionsName} {
...
set file [open "| /usr/bin/rsh -nl $remote(user) $remote(host) cat /proc/net/arp"]
fileevent $file readable {set ::arp::remote(busy) 0}
vwait ::arp::remote(busy)
...
}
</pre></td><tr></table>
Since the file to be read sits in a remote machine on perhaps a slow link, the initialize procedure is blocked until the file becomes readable, but without hanging the user interface, using the Tcl <i>fileevent</i> and <i>vwait</i> commands combination.<br><br>
<li>
<b>Perl</b>: <i>not possible at this time</i>.
<li>
<b>Python</b>: <i>not possible at this time</i>.
</ul>
<p>Similar techniques must be used in such cases within the module update procedure, so that the user interface and any modules running concurrently are not prevented to update. The remote capable moodss modules contain various coding techniques achieving that functionality, which I am sure you can improve on.
<h4><a name="termination"></a>2.4. Termination</h4>
<p>If a procedure named <i>terminate</i> exists in the module namespace, it is invoked when the module is unloaded dynamically. That procedure must take no arguments. For example:
<ul>
<li>
<b>Tcl</b>:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
proc terminate {} {
# do some cleanup
}
</pre></td><tr></table>
<li>
<b>Perl</b>:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
sub terminate() {
# do some cleanup
}
</pre></td><tr></table>
<li>
<b>Python</b>:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
def terminate():
# do some cleanup
pass
</pre></td><tr></table>
</ul>
<h4><a name="variabledata"></a>2.5. Variable data</h4>
<p>The tabular data (variable data) that the module code must update is stored in the <i>data</i> array (same as the module configuration data in <b>Tcl</b>, named after the module configuration data hash in <b>Perl</b>, and after the module configuration data list in <b>Python</b>).
<p>In case of a synchronous module, the core invokes the module <i>update</i> procedure (which obviously must exist) when it is time to refresh the data display (tables and possibly graphical viewers). At this time, the update procedure may update the tabular data straight away (synchronous operation) or launch a request for later data update (asynchronous operation (<i>only available in Tcl</i>)).
<p>In case of an asynchronous module, variable data may be updated at any time. The <i>update</i> procedure may not exist.
<p>For all module types, it actually does not matter when the data is updated. The core will know that fresh data is available when the <i>updates</i> array member is set (actually incremented as it also serves as a counter for the number of updates so far).
<br>It is the module programmer's responsibility to increment this counter right after all tabular data has been updated.
<p>For example, retrieving information for the processes running on a machine is a local operation that can be achieved in a reasonably small amount of time. In such a case, data would be updated immediately and the <i>updates</i> variable incremented at the same time.
<br>But if the data has to be retrieved from across a network, waiting for it to come back would cause a delay that the user would certainly notice, as the application would not respond to mouse or keyboard input during the whole time that it would take to fetch the whole data. In such cases, it is easier to let the update procedure return immediately without setting the <i>updates</i> variable, which would be incremented at a later time, only when the data would become available.
<p>For example, in Tcl, when waiting for data to come across a network connection, the <i>fileevent</i> command could be used on a non blocking channel, where the script to be evaluated when the channel becomes readable would increment the <i>updates</i> array member.
<p>In Perl, this is not yet possible, since the Perl interpreter is dormant outside of the <i>update()</i> function call (<i>a solution to this problem is being studied</i>).
<p>In Python, it is possible to use threads, but at this time, there is no way to pass the information back to the Tcl interpreter outside of the Tcl interpreter driven call of the <i>update</i> function (<i>a solution to this problem is being studied</i>).
<ul>
<li>
<b>Tcl</b>: The tabular data array index is the <i>row</i> number followed by the <i>column</i> number separated by a <i>comma</i>. Example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
proc update {} {
variable data
array set data "
0,0 john 0,1 1234 0,2 4567 0,3 cc
1,0 william 1,1 8901 1,2 2345 1,3 xedit
2,0 anny 2,1 6789 2,2 0123 2,3 ps
4,0 peter 4,1 4567 4,2 8901 4,3 ls
6,0 laura 6,1 2345 6,2 6789 6,3 emacs
3,0 robert 3,1 1234 3,2 5678 3,3 top
"
incr data(updates)
}
</pre></td><tr></table>
<li>
<b>Perl</b>: Data is kept in an array (of rows) of arrays (of column cells). Example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
our @data;
...
sub update() {
@data = (
['john', 1234, 4567, 'cc'],
['william', 8901, 2345, 'xedit'],
['anny', 6789, 0123, 'ps'],
['peter', 4567, 8901, 'ls'],
['laura', 2345, 6789, 'emacs'],
['robert', 1234, 5678, 'top']
);
}
</pre></td><tr></table>
<li>
<b>Python</b>: Data is kept in a list (of rows) of lists (of column cells). Example:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
data = []
...
def update():
global data
data = [
['john', 1234, 4567, 'cc'],
['william', 8901, 2345, 'xedit'],
['anny', 6789, 0123, 'ps'],
['peter', 4567, 8901, 'ls'],
['laura', 2345, 6789, 'emacs'],
['robert', 1234, 5678, 'top']
]
</pre></td><tr></table>
</ul>
<p>The column number must start from 0 up to the total number of columns minus 1 (no holes are allowed in the column sequence).
<br>The row number can take any positive integer value (between 0 and 2147483647) and be defined in any order, as long as it is unique during the lifetime of the module data. If a new row is created, it must take a value that was never used: the index of a row that has disappeared cannot be reused. Row numbers need not be consecutive.
<p>When all rows (or only those table cells that have changed) have been updated, the <i>updates</i> member array must be incremented so that the core knows that it can update the table data display.
<p>The Tcl random module source code can be made to function asynchronously: please look into the random.tcl file.
<h4><a name="rows"></a>2.6. Row numbering</h4>
<p>The module data is internally organized in a 2 dimensional table with the traditional rows and columns. While column numbers must imperiously start from 0 and be consecutive, row numbers can take any value between 0 and 18446744073709551615 (any values allowed for a 64 bit unsigned integer) (<i>even on 32 bit hardware, as long as Tcl/Tk 8.4 or above is used</i>), and need not be consecutive.
<br>However, row numbers need to be unique and the module code must maintain a one to one relationship between row numbers and table row keys. In other words, as in a database table, the module internal table has an associated key: the column or columns that uniquely identify a row of data.
<br>This is very important in saved dashboards, as viewers, thresholds, database history, ... rely on the internal row/column pair to identify a cell. If the next time the module is started, the row number points to data belonging to another monitored item, any viewers containing data cells with that row number, for instance, would display unexpected results.
<p>For example, in a module monitoring the traffic between computers in an enterprise Intranet, the table key will be formed by the 2 columns: source and destination hosts. It is vital to maintain a reliable mapping between the table key and the row numbers, function giving the same result every time the module is loaded. In the computer traffic module, the IP addresses of the source and destination computers could be used to form a unique row number, by concatenating the two 32 bit IP addresses to form a unique 64 bit row number.
<p>As an example of what not to do, consider a module displaying the statistics per hard disk drive on a UNIX computer, with the row numbers assigned in sequence. For example, on a machine classically using 2 SCSI disks and an IDE CD-ROM device:
<p><table border="1">
<tr><th>row number</th><th>disk device</th><th>kilobytes/second written</th><th>kilobytes/second read</th></tr>
<tr><td>1</td><td>hdd</td><td>0</td><td>0</td></tr>
<tr><td>2</td><td>sda</td><td>0</td><td>1214.2</td></tr>
<tr><td>3</td><td>sdb</td><td>511</td><td>7899</td></tr>
</table>
<p>With this module loaded, a dashboard with a graphical viewer displaying the written data rate for the <i>sda</i> disk (data cell: (<b>2</b>,<b>2</b>) (row = 2, column = 2)) is created and saved.
<p>One day, an IDE disk drive is added to the machine. After power-up, the internal data table then becomes:
<p><table border="1">
<tr><th>row number</th><th>disk device</th><th>kilobytes/second written</th><th>kilobytes/second read</th></tr>
<tr><td>1</td><td>hda</td><td>7654</td><td>0</td></tr>
<tr><td>2</td><td>hdd</td><td>0</td><td>0</td></tr>
<tr><td>3</td><td>sda</td><td>0</td><td>67</td></tr>
<tr><td>4</td><td>sdb</td><td>1580</td><td>0</td></tr>
</table>
<p>Now, next time the dashboard is reloaded in moodss, the graphical viewer will continue displaying the data cell (<b>2</b>,<b>2</b>) value, which is now the written data rate for the wrong disk, <i>hdd</i>, which since it is a CD-ROM drive, will remain at zero for ever.
<br>Of course, the same error could occur on a threshold entry, data archived in a SQL database, ...
<p>As a module developer, you must insure that row numbers are positive. Be careful with the <i>incr</i> and <i>expr</i> commands in Tcl, as they may generate negative row numbers, for example after incrementing 2147483647 with Tcl version 8.3. If you want to make sure to generate 32 or 64 bit numbers, the <i>format</i> command is a safe way:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
set row [format %u $value] ;# 32 bits unsigned integer
set row [format %lu $value] ;# 64 bits unsigned integer
</pre></td><tr></table>
<p>Various techniques can be used to generate unique row numbers. For example, let us have a look at some of the modules included in the moodss distribution:<ul>
<li><i>arp</i> and <i>route</i>: since the internal table key is an IP <b>address</b>, it is simply converted to a 32 bit row number.
<li><i>disks</i> and <i>diskstats</i>: as defined in the Linux kernel, a unique 32 bit row number can be generated by shifting the <b>major</b> device number 20 bits to the left and OR'ing with the <b>minor</b> device number.
<li><i>usb</i>: a unique 32 bit row number is generated by shifting the <b>bus</b> device number 16 bits to the left and OR'ing with the <b>device</b> number.
<li><i>interrupts</i>: the row number is simply the <b>interrupt</b> number, or a 32 bit number generated by concatenating the characters forming the short <b>name</b> of the interrupt (length less than or equal to 4 bytes).
<li><i>pci</i>: the 32 bit row number is generated by shifting the <b>bus</b> device number 8 bits to the left, OR'ing with the <b>device</b> number shifted 3 bits to the left and OR'ing with the <b>function</b> number.
<li><i>sensors</i>: the row number is a 64 bit number generated by concatenating the characters forming the <b>name</b> of the sensor data (length less than or equal to 8 bytes).
<li><i>kernmods</i>, <i>mounts</i>, <i>netdev</i>, <i>ping</i>, <i>psbyname</i> and <i>psbyuser</i>: a unique row number is drawn from the row <b>name</b> entry (module, mount point, network device, host name or address or process name) by applying a 64 bit hash algorithm on the character string.
<li><i>snmp</i> and <i>smithy</i>: when monitoring a table (in the SNMP sense, described in a MIB), the 64 bit row number is generated by hashing the list of 32 bit words forming the sub-identifier part of the table key.
</ul>
<p>The <i>persistent</i> data member should be set to <i>1</i> once the module implementation guarantees that row numbers are reliably generated (see <a href="#configuration">configuration</a> section).
<p><i><b>Note</b>: a <a href="#hash">hash library</a> is included in the distribution.</i>
<h4><a name="internationalization"></a>2.7. Internationalization</h4>
<p>The moodss GUI core is multi-lingual, thanks to Tcl internationalization support. Modules can also support different languages, for data column titles, help messages, module help, ... All that is required is the following code near the beginning of the module source code:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package require msgcat
namespace import msgcat::*
mcload .
</pre></td><tr></table>
<p>Then any string candidate for internationalization needs be coded as:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
set message [mc "Good bye"] ;# = "Au revoir" in French
</pre></td><tr></table>
if the <i>fr.msg</i> file is present in the module directory and contains:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
msgcat::mcset fr "Good bye" "Au revoir"
</pre></td><tr></table>
and, for example, the <b>LANG</b> environment variable was set to <b>fr</b>.
<p><i><b>Note</b>: the <b>.msg</b> files must be encoded in <b>UTF-8</b>.</i>
<p>A different technique must be used for the module HTML help file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
if {\
([info exists ::env(LC_ALL)] && [string match fr* $::env(LC_ALL)]) ||\
([info exists ::env(LANG)] && [string match fr* $::env(LANG)])\
} {
set file [open random-fr.htm] ;# (written in French)
} else {
set file [open random.htm] ;# (written in English)
}
</pre></td><tr></table>
<p>Also note that the HTML file must be encoded in <i>ISO8859-1</i> for <i>French</i>, <i>EUC-JP</i> for <i>Japanese</i>, ... (please refer to the Tcl documentation for more information).
<h3><a name="installation"></a>3. Installation</h3>
<p>A module is a package in the Tcl sense. When writing a module, you must then provide a <i>pkgIndex.tcl</i> file along with the module code file, placed in the module directory. The <i>pkgIndex.tcl</i> file is very simple, as the following example shows:
<ul>
<li>
<b>Tcl</b>: the line below says that if the <i>random</i> package is needed, the core should source the <i>random.tcl</i> module source code from the directory where it was installed. <i>1.1</i> is the version number for the package, identical to the one specified in the <i>package provide</i> instruction in the module source code file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package ifneeded random 1.1 "source [file join $dir random.tcl]"
</pre></td><tr></table>
<li>
<b>Perl</b>: the line below says that if the <i>Random</i> package is needed (note the capitalized name required for Perl packages), the core should load the <i>Random.pm</i> module source code into its own Perl embedded interpreter from the directory where it was installed. <i>1.1</i> is the version number for the package, identical to the one specified in the <i>$VERSION</i> assignment in the module source code file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package ifneeded Random 1.1 "source [file join $dir Random.pm]"
</pre></td><tr></table>
<li>
<b>Python</b>: the line below says that if the <i>randpy</i> package is needed, the core should load the <i>randpy.py</i> module source code into its own Python embedded interpreter from the directory where it was installed. <i>1.1</i> is the version number for the package, identical to the one specified in the <i>__version__</i> assignment in the module source code file:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package ifneeded randpy 1.1 "source [file join $dir randpy.py]"
</pre></td><tr></table>
</ul>
<p>Modules can be installed at any valid place that the Tcl core allows (look at the <i>pkg_mkIndex</i> manual page for more information).
<p>When you unpack moodss, you will find the sample modules in sub directories. The current directory (.) is appended to the <i>auto_load</i> global list variable so that sample modules can be found when moodss is run from the unpacking directory.
<p>For example, if you unpacked moodss in <i>/home/joe/moodss-X.x/</i>, you will find the random module package in <i>/home/joe/moodss-X.x/random/</i> so that the following will work:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
$ cd /home/joe/moodss-X.x/
$ wish moodss random
</pre></td><tr></table>
<p>You can install your new modules in the default location: <i>/usr/local/lib/</i> on UNIX. For example, if you move the files in <i>/home/joe/moodss-X.x/random/</i> to <i>/usr/local/lib/random/</i>, moodss will still be able to find the <i>random</i> module (again, look at the <i>pkg_mkIndex</i> manual page for more information).
<p><i><b>Note</b>: when the <b>--debug</b> option is used in the command line, moodss will allow loading modules that are not in a moodss directory (a directory containing the "moodss" string in its full path).</i>
<p>Please take a look at the INSTALL file for the latest information on how to install the moodss application itself.
<h3><a name="displaying"></a>4. Displaying messages</h3>
<i><b>Note</b>: this functionality is not yet available for Python modules.</i>
<p>You may want to inform the user of the module activity. You may use the message area (called the messenger, across the bottom of the application main window) through the following API:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
<b>pushMessage</b> "error: message..."
<b>popMessage</b>
<b>flashMessage</b> "warning: message..." <i>numberOfSeconds</i>
</pre></td><tr></table>
<p>Your message can be any kind of string (1 line only: it should fit in a reasonably wide main window), and <i>numberOfSeconds</i> being optional and defaulting to 1. Note that messages sent to the message area are also displayed in the trace module table(s) if it(they) exist(s) (see below). In such cases, popping messages has no effect on trace module tables.
<ul>
<li><b>pushMessage</b> puts the string passed as parameter to the top of the messenger internal stack and displays it immediately as: "<i>identifier: string</i>", where identifier is either the module identifier string set in the module implementation or by default the module name followed by its instance number if there are several instances of the same module.
<li><b>popMessage</b> discards the string at the top of the internal messenger stack and immediately displays the next string in the stack, or nothing if there is none.
<li><b>flashMessage</b> is equivalent to a push followed by a pop a number of seconds later. A flash always discards a yet active flash if there is one.
</ul>
<p>One should use the message area facilities with discretion, as it is already used by the application core for informing the user of modules loading, initialization, updates, context sensitive help, ... My advice is to use it for important messages or errors pertinent to the module itself.
<p>The module name should not appear at the beginning of the messages as it is automatically prepended internally (see <i>pushMessage</i> above).
<br>I suggest using the importance level (as in <a href="#menus.edit.thresholds">thresholds</a>) as the header. Possible values are, in increasing importance order: <i>debug</i>, <i>info</i>, <i>notice</i>, <i>warning</i>, <i>error</i>, <i>critical</i>, <i>alert</i>, <i>emergency</i>. Look for examples in included modules.
<p>You also have the option of using trace tables (also see <a href="#trace">trace</a> module) through the following API:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
<b>traceMessage</b> "critical: message..."
</pre></td><tr></table>
<p>The difference is that messages from the module are displayed in the trace module tables if they exist (see <a href="#menus.view.trace">trace</a>, there can be more than 1 trace module loaded), remain visible to the user for a longer period of time, and can be multi-line.
<p>Finally, displaying separate (<i>toplevel</i>) message windows using Tk is of course always possible (if you load Tk in the module). Keep in mind that the core, not being aware of the module event that has taken place, will continue to invoke the module <i>update</i> procedure (unless the module is asynchronous, of course). In such a case, you may want to use an internal (to the module) busy flag so that the update procedure immediately returns until the user acknowledges the informational message. Other strategies are of course possible <i>(let me know if you have one that you think should appear here as an example)</i>.
<h3><a name="errors"></a>5. Error handling</h3>
<p>In order to allow clean error handling with module loading either by command line arguments or dynamically while the application is running, the following rules need be followed:
<ul>
<li>in the module body, use:<ul>
<li><b>Tcl</b>: the <i>error</i> command.
<li><b>Perl</b>: the <i>die</i> function.
<li><b>Python</b>: the <i>raise</i> statement.
</ul>
<li>while in the module initialize procedure or function, use:<ul>
<li><b>Tcl</b>: the <i>error</i> command.
<li><b>Perl</b>: the <i>die</i> function.
<li><b>Python</b>: the <i>raise</i> statement.
</ul>
<li>thereafter, either the short messages facility (for Tcl only, see <a href="#displaying">Displaying messages</a>) or printing to standard output or error must be used, possibly exiting in case of fatal errors
</ul>
<p>Specifying the module name in the messages is not necessary as the core handles it.
<h3><a name="daemon"></a>6. Daemon specifics</h3>
<p>Before moodss was packaged with a daemon (moomps), it was natural for such a GUI application to immediately react on module errors: when there is an error in a module initialization stage, the error is displayed in a dialog box and loading the module is aborted. Since the user is in front of the screen, he can try to fix the problem right away.
<p>Now if the same module is used by the moomps daemon, in case of initialization error when the daemon is started unattended, the error message is logged but the module is never loaded. This is a problem if the error cause is a temporary communication breakage with a remote host, for example.
<p>One solution would be for the moomps daemon to keep retrying loading a module until successful, but that may cause more problems than it solves, especially if the module loads a binary library or triggers some other mechanism which may not accept such forceful treatments (possibly when using secure channels, encryption, ...), or if the module contains bugs that prevents clean unloading, which may result in moomps hanging or crashing.
<p>The module knows best if, when and how to retry its initialization, so if the module supports the <i>--daemon</i> option (no arguments), it is automatically set by moomps for the purpose of letting the module know whether it should keep trying to reinitialize on failures, or more generally do something different in daemon mode.
<h3><a name="perl"></a>7. Perl</h3>
<p>Moodss modules can also be written in the Perl language, and even internally use Perl threads for potentially processor intensive or blocking operations.
<p>An embedded Perl interpreter is used for each loaded module, thus achieving complete independence between modules.
<p>The <i>tclperl</i> library is required (available on my homepage), at least version 3.1 if Perl threads are to be used.
<p>As described in the section about <a href="#displaying">displaying messages</a> in Tcl, such functionality has become available to Perl modules starting with moodss version 19.1. The API is very similar:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
<b>pushMessage</b>("error: message...");
<b>popMessage</b>();
<b>flashMessage</b>("warning: message...", <i>numberOfSeconds</i>);
<b>traceMessage</b>("critical: message...");
</pre></td><tr></table>
<p>Also, managing timers (timeouts, ...) is possible without blocking the core (<i>sleep()</i> would block the GUI, for example), by using the following function, which causes the core to invoke a piece of code (as in Perl <i>eval()</i>) back in the module Perl interpreter, after a specified number of milliseconds:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
<b>after</b>(<i>milliseconds</i>, 'Perl code...');
</pre></td><tr></table>
<p><i><b>Note</b>: an example is given in the Random module, in order to implement the asynchronous mode.</i>
<p>Finally, when using Perl threads in a Perl module, which is the only way to ever prevent blocking the core (for example, the GUI could be otherwise blocked waiting for a connection to reestablish with a down computer), the following function must be used within a thread:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
<b>yield</b>('updated');
</pre></td><tr></table>
<p>It gives control back to the core once a thread is finished working. In turn, the core calls the <i>updated()</i> function in the Perl module. Only the '<i>updated</i>' function is supported at this time. For example, Perl queues can be used to pass data from the threads to the Perl module interpreter. This functionality has become available starting with moodss 19.1.
<br><i><b>Note</b>: this is required as the parent Perl interpreter is generally no longer running at the time the thread has done its job, and thus needs to be awaken by the application core, which is always running.</i>
<p>Further documentation for programming Perl modules can be found in the fully commented <i>Random.pm</i> and <i>Threaded.pm</i> modules source files.
<h3><a name="python"></a>8. Python</h3>
<p>Moodss modules can also be written in the Python language.
<p>An embedded Python interpreter is used for each loaded module, thus achieving complete independence between modules.
<p>The <i>tclpython</i> library (version 3.0 or above, for Python 2.2 or above) is required (available on my homepage as source and Red Hat rpms).
<p>Complete documentation for programming Python modules can be found in the fully commented <i>randpy.py</i> module source file.
<h3><a name="threadsevents"></a>9. Threads and events programming</h3>
<p>The Tcl language allows non blocking implementations in 2 ways:<ul>
<li>using events on sockets, with callbacks
<li>using threads (using the Tcl Thread extension, only available for Tcl 8.4 and above)
</ul>
<p>Non blocking code is very useful when coding modules, as it prevents the user interface (the moodss core) from being hung when a module, for example, cannot reach a remote host. Such coding techniques are therefore recommended for a graphical user interface, which should stay responsive at all times.
<br>Unfortunately, handling all the different cases and errors is not easy, and coding using threads or events in Tcl involves 2 different techniques.
<p>That is why the object oriented <b>line task</b> class (linetask.tcl file) was created and is included in the moodss source code. It allows, via a common interface, the transparent handling of communication with a data pipe, uses threads if available, or asynchronous event handling otherwise.
<br>It is used in several modules, and especially in the pci module (<a href="../pci/pci.htm">documentation</a>), where all the code is thoroughly commented and can be used as a reference for all remote capable modules that need to communicate with a remote machine via a pipe.
<p>Note that threads are used in priority if available, as they are guaranteed to never block the main process, which is still not the case using events, as they can hang, for example, on a non responding DNS query.
<h3><a name="utilities"></a>10. Utilities</h3>
<h4><a name="hash"></a>10.1. Hash library</h4>
<p>You may find the <i>hashes.tcl</i> file (in the <i>packlibs</i> sub-directory), which includes a couple of unsigned 64 bit hash functions for hashing strings and lists of 32 bit integers.
<p>The algorithm was chosen after a fair amount of testing, against other well known implementations. It produces very good results, especially considering its simplicity. It is a slightly modified version of the Tcl internal C implementation.
<p>In order to use it inside a Tcl module, just include the following line early in the code:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package require hashes
</pre></td><tr></table>
<p>You may then invoke the following procedures, which return a 64 bit unsigned integer (between 0 and 18446744073709551615) either on a string of characters or a list of 32 bit integers:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
hash64::string <i>string</i>
hash64::numbers32 <i>list</i>
</pre></td><tr></table>
<p><i><b>Note</b>: for <b>Python</b> and <b>Perl</b> modules, I suggest porting the Tcl implementation from the <b>hashes.tcl</b> file, which should be quite easy.</i>
<h4><a name="linetask"></a>10.2. Non-blocking channel library</h4>
<p>Modules that can communicate with a remote computer, and that use a TCP based protocol (as the <i>rsh</i> and <i>ssh</i> commands), may hang in case of communication problems. Unfortunately, such problems also cause the core (the moodss GUI or the moomps daemon) to hang until the timeout elapses or the communication error is caught or solved.
<p>Fortunately, there are two ways to circumvent those problems in the Tcl language: event based programming or threads. Unfortunately, depending on the Tcl/Tk version used, threads facilities might not be available, and coding for either events or threads rapidly becomes cumbersome.
<p>That is why the <i>linetask</i> library was made part of the moodss distribution, and is used in several included modules. This library, implemented as a <i>stooop</i> class, can be found in the <i>linetask.tcl</i> file in the <i>packlibs</i> sub-directory. In order to use it in a module, include the following line early in the code:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package require linetask 1
</pre></td><tr></table>
<p>The <i>lineTask</i> class provides a unified implementation of a non-blocking channel library, by internally using either threads or events, while preserving the same API in both cases. The preferred method is the <i>threads</i> implementation, as even in the <i>events</i> case, timeouts caused by a failing remote host or network may still hang the module process.
<p>In order to use threads, the Tcl threads library (binary compiled along the Tcl/Tk installation) and the included thread worker class implementation (see <i>threads.tcl</i> in the <i>packlibs</i> sub-directory) are needed:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
package require Thread 2.5
package require threads 1
package require linetask 1
</pre></td><tr></table>
<p>Then, if using <i>ssh</i> to connect to the remote host, pseudo-tty allocation is disabled as obviously the session is not interactive and the standard error channel redirected to the standard output so errors on the remote end can be detected and reported. A callback procedure for reading data lines from the remote shell is specified, while access is specified as bi-directional since scripts will be sent (written) to the remote host shell, translation is UNIX compatible, threaded code is enabled and command is not to be immediately executed:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
set command "ssh -T -l $remote(user) $remote(host) 2>@ stdout"
set remote(task) [new lineTask\
-command $command -callback pci::read\
-begin 0 -access r+ -translation lf -threaded 1\
]
</pre></td><tr></table>
<p>Later on, when the module is requested to update its data by the core, the task command is executed and a script initiating the data retrieval sent to the remote shell:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
lineTask::begin $remote(task)
lineTask::write $remote(task) "/sbin/lspci | tr '\\n' '\\v'"
</pre></td><tr></table>
<p>Note that the task is line oriented (hence the <i>lineTask</i> name), so all data lines are gathered into one using the <i>tr</i> command and the vertical tab character (any rare control character would probably do) as a separator. Having the data packed in a single line removes potential line buffering problems from the remote shell.
<p>Until the data becomes available from the remote host, the core (moodss or moomps) can continue servicing user requests and update other modules. When the data arrives, the callback is invoked as follows:
<table bgcolor="#DFDFDF" width="100%"><tr><td><pre>
proc read {line} {
process [split [string trimright $line \v] \v]
}
</pre></td><tr></table>
<p>The packed data line is transformed back into a list of data lines before processing (the process procedure is an internal module procedure that formats the data, updates the module data array and finally lets the core know that new data has arrived).
<p>The <b>pci</b> module contains a fully commented reference implementation of data retrieval and processing from a remote host, including <i>events</i> or <i>threads</i> programming automatic switch depending on Tcl/Tk core capabilities, user selectable <i>ssh</i> or <i>rsh</i> communication with remote host and reliable error handling and recovery.
<p><i><b>Note</b>: for <b>Python</b> and <b>Perl</b> modules, such facility is not available at this time.</i>
<h3><a name="tips"></a>11. Tips and tricks</h3>
<h4><a name="singlerow"></a>11.1. Single row tables</h4>
<p>When data to be displayed is an array of unrelated values, the only solution is to organize the data in a table with a single row and 1 column per value (which does not prevent the use of several views for displaying the data).
<p>In such a case, no index column is required and as a matter of fact rather gets in the way. The trick is to use an empty column 0, use as index by the core by default, and to prevent it from being displayed by using 1 or more views.
<p>Please look at the <i>cpustats</i> module code for a working example.
<h4><a name="void"></a>11.2. Void values as numbers</h4>
<p>In the module data array, you cannot leave or set a numeric cell empty when no data is available. Use the ? character (also used in statistics tables) instead.
</body>
</html>
|
