1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<!--#include file="Head.html"-->
<body bgcolor="#FFFFFF" leftmargin=0 topmargin=0 text="black" link="#0000FF"
vlink="#000000" alink="#000000">
<!--#include file="Table.html"-->
<table>
<tr>
<!--#include file="SideBar.html"-->
<td width="550" valign="top" bgcolor="#FFFFFF">
<div align="left">
<a name="general"></a><font size=5>Dart Client Usage - Concepts</font>
<p>
The Dart client operates under a number of <em>Models</em>:
<ul>
<li><b>Nightly</b> - Build/test sequences in lockstep with all other nightly builds. Nightly builds are the backbone of the Dart SQA process. Nightly builds provide the best opportunity to compare build/test sequences across multiple hardware or software configurations.
<li><b>Continuous</b> - Build/test sequences started whenever something changes in the repository. Continuous builds allow the developers to monitor the impact of changes made to the software throughout the day.
<li><b>Experimental</b> - Developer driven testing. This model is used by any developer to test their local copy of the software. Results from either form of experimental build/test sequences can be submitted to the nightly dashboard for all developers to see.
</ul>
Under each model, the Dart client accepts a number of commands.
The <em>Commands</em> for Nightly, Continuous and Experimental models are
<ul>
<li><b>DashboardStart [date]</b> - Starts a dashboard for the given date.
Yesterday is default, only for nightly. Note that it triggers an Update automatically. DashboardStart only needs to be run on the Dart server.
<li><b>Start</b> - Create a directory to store the build/test results.
<li><b>Update</b> - For a Nightly build, update the code to the nightly timestamp. For a Continuous or Experimental build, update the code to reflect the current contents of the repository. For Experimental builds, Update reports on locally modified files. IMPORTANT: in the Nightly model the cvs update is done against the previous day's repository snapshot, not against the latest sources (Continuous or Experimental models). For Continuous builds, if no changes are detected in the repository, all remaining commands are skipped (no build is done and no tests are run) and no data is transmitted to the Dart server.
<li><b>Configure</b> - Configure the build (could run CMake or a configure script).
<li><b>Build</b> - Build the software and place reports in the proper directory (created by the Start command).
<li><b>Test</b> - Test the software and place reports in the proper directory (created by the Start command).
<li><b>Coverage</b> - Summarize the source code coverage of the test cases.
<li><b>Purify</b> - Summarize dynamic memory usage of the test cases. Search for memory leaks, array bound reads, etc..
<li><b>Submit</b> - Submit the last set of reports to the Dart server.
<li><b>DashboardEnd [date]</b> -- Ends a dashboard for the given date.
Yesterday is default, only for nightly. DashboardEnd only needs to be run on the Dart server.
</ul>
<br><p>
<a name="CMake"></a><font size=5>Dart Client Usage - CMake</font>
<p>If Dart is configured to use CMake, then CMake will produce a number of make targets (Unix) or project files (Windows) to control Dart.
<p>
<a name="unix"></a><h3>Unix Make Targets</h3>
<ul>
<li><b> make Nightly</b> - This make target performs a <em>Start</em>, <em>Update</em>, <em>Build</em>, <em>Test</em>, <em>Submit</em> for a nightly build.
<li><b> make NightlyStart</b> - This make target performs a <em>Start</em> for a nightly build.
<li><b> make NightlyUpdate</b> - This make target performs an <em>Update</em> for a nightly build.
<li><b> make NightlyBuild</b> - This make target performs a <em>Build</em> for a nightly build.
<li><b> make NightlyTest</b> - This make target <em>Test</em>'s a nightly build.
<li><b> make NightlyCoverage</b> - This make target computes <em>Coverage</em> for a nightly build.
<li><b> make NightlyPurify</b> - This make target computes <em>Purify</em> (memory leaks, array bound reads, etc.) for a nightly build.
<li><b> make NightlySubmit</b> - This make target <em>Submit</em>'s nightly build/test results to the Dart server.
</ul>
<br>
<ul>
<li><b> make Continuous</b> - This make target performs a <em>Start</em>, <em>Update</em>, <em>Build</em>, <em>Test</em>, <em>Submit</em> for a continuous build.
<li><b> make ContinuousStart</b> - This make target performs a <em>Start</em> for a continuous build.
<li><b> make ContinuousUpdate</b> - This make target performs an <em>Update</em> for a continuous build.
<li><b> make ContinuousBuild</b> - This make target performs a <em>Build</em> for a continuous build.
<li><b> make ContinuousTest</b> - This make target <em>Test</em>'s a continuous build.
<li><b> make ContinuousCoverage</b> - This make target computes <em>Coverage</em> for a continuous build.
<li><b> make ContinuousPurify</b> - This make target computes <em>Purify</em> (memory leaks, array bound reads, etc.) for a continuous build.
<li><b> make ContinuousSubmit</b> - This make target <em>Submit</em>'s continuous build/test results to the Dart server.
</ul>
<br>
<ul>
<li> <b>make Experimental</b> - This make target performs a <em>Start</em>, <em>Build</em>, <em>Test</em> for an experimental build.
<li><b> make ExperimentalBuild</b> - This make target performs a <em>Start</em> and <em>Build</em> for an experimental build.
<li><b> make ExperimentalTest</b> - The make target only performs a <em>Test</em> of an experimental build.
<li><b> make ExperimentalCoverage</b> - This make target computes <em>Coverage</em> for an experimental build.
<li><b> make ExperimentalSubmit</b> - This make target <em>Submit</em>'s an experimental build.
</ul>
<br>
<ul>
<li><b> make NightlyDashboardEnd</b> - This make target rolls up a dashboard. Summarizing any information submitted since the last time the dashboard was generated.
<li><b> make NightlyDashboardStart</b> - This make target opens a dashboard for submissions.
</ul>
<br>
<p>
<a name="win"></a><h3>Windows MS Developer Projects</h3>
<ul>
<li> <b> Experimental.dsp</b> - This project performs a <em>Start</em>, <em>Build</em>, <em>Test</em> for an experimental build.
<li><b> ExperimentalSubmit.dsp</b> - This project <em>Submit</em>'s an experimental build.
</ul>
<br><p>
<a name="tcl"></a><font size=5>Dart Client Usage - Tcl</font>
<p>Dart can controlled directly using the Tcl scripts in Dart/Source/Client. These scripts are run from your project's build directory and have the form
<pre>
tclsh Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl Model Command [Command ...]
</pre>
The file <b>DartConfiguration.tcl</b> must exist in the project build directory. If you are using Dart with CMake, CMake will construct this file automatically, using Dart/Source/Client/Utility.conf.in as a template. If CMake is not being used, DartConfiguration.tcl must be constructed by hand.
<br><br><br><p>
<a name="tcl"></a><font size=5>Setting up a Nightly Client</font>
<p> Each Dart client should have a separate checkout of a
project's source code and a separate build tree. Configure the
build as you normally would. Define the SITE and BUILDNAME via
CMake (or in DartConfiguration.tcl if not using CMake). Also make
sure the CVSCOMMAND, MAKECOMMAND, TCL_TCLSH are defined
properly. When using Microsoft Visual Studio as your compiler, the
MAKECOMMAND will specify which build configuration (Release,
Debug, etc.) to use. Set MAKECOMMAND to use the build
configuration you want. Dart will search for test executables in
a prescribed order (Release, Debug, MinSizeRel,
RelWithDebInfo). Dart will run a test from the first configuration
it finds. Dart clients, therefore, should only have a single
configuration built. All other configurations should be
empty. Future versions of Dart may allow the client to specify
which configuration to run tests from. Finally, if your testing
machine is behind a firewall, you may have set the environment
variables HTTP_PROXY and HTTP_PROXY_PORT.
<p> Once a build is configured, schedule a job to run your nightly build. The scheduled time can be anytime after the NIGHTLY_START_TIME for your project. Your scheduled job should set any necessary environment variables you need to build and run. Running a nightly build/test sequence can be accomplished with the following commands
<pre>
cd ProjectNightlyBuildDirectory
tclsh Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \
Nightly Start Update Configure Build Test Submit
</pre>
If you would like to submit results to the Dartboard as commands
finish, you can break the command into several Dart commands. The
following will submit the update and build results to the Dart server,
then run the tests and submit those results.
<pre>
cd ProjectNightlyBuildDirectory
tclsh Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \
Nightly Start Update Configure Build
tclsh Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \
Nightly Test Submit
</pre>
<br><p>
<a name="tcl"></a><font size=5>Setting up a Continuous Client</font>
<p> A Continuous client is setup similar to a Nightly client. It should have its own source checkout and its own build area. Whereas a Nighly client is driven by a scheduled job, a Continuous client is usually started once and runs forever. A script to run a client in Continuous mode would look like:
<pre>
cd ProjectContinuousBuildDirectory
while (1)
tclsh Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \
Continuous Start Update Configure Build Test Submit
sleep 300
end
</pre>
Here, the Dart commands are inside a "while" loop and the client is
instructed to sleep for 5 minutes after completing a build/test
sequence. Also note that we do not break the Dart commands into
several command sequences like we can do for a Nightly. The reason is
that an "Update" on a Continuous build will terminate the Dart client
when nothing has changed in the repository. This avoids building and
running tests when nothing has changed. This feature is only
available when a Dart Continuous client runs all the Dart commands in
one operation.
<p>Hint: Turn off compiler optimisation with continuous testing. Unless your
tests are very compute heavy, this continuous build cycle is dominated by
the compiler and optimisation will just make it take longer. If compute
resources are scarce, the same suggestion applies to nightly builds -
however, it is useful to check for errors with the optimisation settings you
will use to build your application for normal use. </p>
<p>
</div>
</td>
</tr>
</body>
|
