1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193
|
<p>Tup handles dependencies a little differently from other build systems. In this example, we'll show how the dependencies that you specify in a Tupfile work together with the dependencies determined automatically during the program's execution.</p>
<p>First setup a test directory with a Tupfile and shell script:</p>
<pre>
<span class="prompt">$</span> tup init tup_test2
<span class="prompt">$</span> cd tup_test2
</pre>
<span class="fileheader">test.sh</span>
<pre class="code">#! /bin/sh
echo "Output from test.sh"
</pre>
<span class="fileheader">Tupfile</span>
<pre class="code">: |> ./test.sh > %o |> output.txt
</pre>
<pre>
<span class="prompt">$</span> chmod +x test.sh
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.006s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[ 1/1 ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] ./test.sh > output.txt
[ tup ] Updated.
<span class="prompt">$</span> cat output.txt
Output from test.sh
</pre>
<p>So our simple script ran and created the <span class="filename">output.txt</span> file. When will tup decide to run the script again to create a new output? The answer is simple: whenever the <span class="filename">test.sh</span> script changes! Watch as <span class="cmd">tup</span> does not re-run the script until the script is touched:</p>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.000s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] No commands to execute.
[ tup ] Updated.
<span class="prompt">$</span> touch test.sh
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.005s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] ./test.sh > output.txt
[ tup ] Updated.
</pre>
<p>We can change the script to read from other files, as well. Suppose we create a new text file, called <span class="filename">header.txt</span>, that gets cat'd at the top of the script:</p>
<span class="fileheader">header.txt</span>
<pre class="code">This is the file header
</pre>
<span class="modfileheader">test.sh</span>
<pre class="code">#! /bin/sh
<span class="added">cat header.txt</span>
echo "Output from test.sh"
</pre>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[ 1/1 ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] ./test.sh > output.txt
[ tup ] Updated.
<span class="prompt">$</span> cat output.txt
This is the file header
Output from test.sh
</pre>
<p>Note that we don't need to specify the <span class="filename">header.txt</span> dependency in the Tupfile, but tup will still re-run the script if it changes:</p>
<span class="modfileheader">header.txt</span>
<pre class="code">This is the<span class="added"> *new*</span> file header
</pre>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] No Tupfiles to parse.
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] ./test.sh > output.txt
[ tup ] Updated.
<span class="prompt">$</span> cat output.txt
This is the *new* file header
Output from test.sh
</pre>
<p>Similar to the <a href="ex_a_first_tupfile.html">first Tupfile example</a>, tup has executed the script in such a way that it can track file accesses. Tup sees that the script read from <span class="filename">header.txt</span> and automatically adds the dependency. We can see this from the dependency graph:</p>
<pre>
<span class="prompt">$</span> tup graph --stickies . | dot -Tpng > ~/ex_deps_1.png
</pre>
<div class="img"><img src="ex_deps_1.png"></div>
<p>Both the <span class="filename">header.txt</span> and <span class="filename">test.sh</span> files point to the command that executes the test script. If either of those files change, then tup will re-execute the script. Otherwise there's no need to run it again!</p>
<div style="clear:both;"></div>
<h3>Dependencies on Generated Files</h3>
<p>At this point, the <span class="filename">header.txt</span> and <span class="filename">test.sh</span> files are written by you, the author. Tup refers to these as "normal" files (in the tup source code, this is the TUP_NODE_FILE enum). The <span class="filename">output.txt</span> file is generated by tup, so it is called a "generated" file (TUP_NODE_GENERATED in the source). Generated files are treated a little differently in tup. Let's see what happens if we create a new generated file and try to read from it in the test script:</p>
<span class="modfileheader">Tupfile</span>
<pre class="code"><span class="added">: |> echo "generated text" > %o |> generated.txt</span>
: |> ./test.sh > %o |> output.txt
</pre>
<span class="modfileheader">test.sh</span>
<pre class="code">#! /bin/sh
cat header.txt
<span class="added">cat generated.txt</span>
echo "Output from test.sh"
</pre>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[ 1/1 ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/2 ] echo "generated text" > generated.txt
[ 2/2 ] ./test.sh > output.txt
*** tup errors ***
tup error: Missing input dependency - a file was read from, and was not
specified as an input link for the command. This is an issue because the file
was created from another command, and without the input link the commands may
execute out of order. You should add this file as an input, since it is
possible this could randomly break in the future.
- [8] generated.txt
*** Command ID=6 ran successfully, but tup failed to save the dependencies.
</pre>
<p>Oops, it seems tup didn't like that. The issue here is that tup has no way of knowing that the command to create <span class="filename">generated.txt</span> must run before the <span class="filename">test.sh</span> script. Therefore, it is possible that tup will schedule them in the wrong order (so that <span class="filename">generated.txt</span> isn't created by the time the script runs), or it may even schedule them in parallel.</p>
<p>To give tup the information that <span class="filename">generated.txt</span> must be created first, we simply list it as an input to the test script:</p>
<span class="modfileheader">Tupfile</span>
<pre class="code">: |> echo "generated text" > %o |> generated.txt
: <span class="added">generated.txt</span> |> ./test.sh > %o |> output.txt
</pre>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[ 1/1 ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] ./test.sh > output.txt
[ tup ] Updated.
<span class="prompt">$</span> cat output.txt
This is the *new* file header
generated text
Output from test.sh
</pre>
<h3>Tupfile Dependencies are for Ordering</h3>
<p>Now we will see what happens if you add an input to the Tupfile, but the input goes unused. Let's add another generated file, but the shell script is unchanged so we don't actually read from it:</p>
<span class="modfileheader">Tupfile</span>
<pre class="code">: |> echo "generated text" > %o |> generated.txt
<span class="added">: |> echo "unused text" > %o |> unused.txt</span>
: generated.txt<span class="added"> unused.txt</span> |> ./test.sh > %o |> output.txt
</pre>
<pre>
<span class="prompt">$</span> tup
[ tup ] Scanning filesystem...0.007s
[ tup ] No tup.config changes.
[ tup ] Parsing Tupfiles...
[ 1/1 ] .
[ tup ] No files to delete.
[ tup ] Executing Commands...
[ 1/1 ] echo "unused text" > unused.txt
[ tup ] Updated.
</pre>
<p>Curiously, the <span class="filename">test.sh</span> script did not execute even though <span class="filename">unused.txt</span> is listed as an input. How then were we able to bypass the earlier error message by adding an input in the Tupfile, if those inputs don't actually cause the commands to execute? Let's take a look at our new graph:</p>
<pre>
<span class="prompt">$</span> tup graph . | dot -Tpng > ~/ex_deps_2.png
</pre>
<div class="img"><img src="ex_deps_2.png"></div>
<p>Here we can see that not all of the input dependency arrows are the same. There are solid lines and dotted lines, as well as filled in arrows and empty arrows. The actual details of how tup handles these are beyond the scope of this example. The important thing to see here is that tup still keeps track of the fact that the shell script has a dependency on the <span class="filename">unused.txt</span> file, but because the file was never actually read from by the script, it cannot possibly have an effect on the output file (this fact is represented by the dotted line). Therefore, tup knows that if <span class="filename">unused.txt</span> changes, the script does not need to run again.</p>
<p>However, the presence of the dotted-line dependency means that if <span class="filename">unused.txt</span> is changed and at the same time <span class="filename">test.sh</span> is changed to read from that file, tup is guaranteed to execute them in the correct order. In this sense, inputs in Tupfiles are only for ordering, and dependencies determined automatically during program execution are used for re-executing commands. For more insight into why this is useful, see the <a href="ex_generated_header.html">generated header example</a>.</p>
|