1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558
|
---
layout: documentation
title: Test encyclopedia
---
<h1>Test encyclopedia</h1>
<p class="lead">An Exhaustive Specification of the Test Execution Environment</p>
<h2>Background</h2>
<p>The Bazel BUILD language includes rules which can be used to define
automated test programs in many languages.</p>
<p>Tests are run using <code><a href="user-manual.html#test">bazel test</a></code>.
Users may also execute test binaries directly. This is allowed but not endorsed, as such
an invocation will not adhere to the mandates described below.</p>
<p>Tests should be <i>hermetic</i>: that is, they ought to access only those
resources on which they have a declared dependency. If tests are not properly
hermetic then they do not give historically reproducible results. This could be a
significant problem for culprit finding (determining which change broke a test),
release engineering auditability, and resource isolation of tests (automated
testing frameworks ought not DDOS a server because some tests happen to
talk to it).<p>
<h2>Objective</h2>
<p>The goal of this document is to formally establish the runtime environment
for and expected behavior of Bazel tests. It will also impose requirements on
the test runner and the build system.
Our intent is to help test authors avoid relying on unspecified
behavior, and thus give the testing infrastructure more freedom to make
implementation changes. We will also take the opportunity to tighten up some
holes which currently allow many tests to pass despite not being
properly hermetic, deterministic, and reentrant.</p>
<p>This document is intended to be both normative and authoritative. If
this specification and the implemented behavior of test runner disagree, the
specification takes precedence.</p>
<h2>Purpose of tests</h2>
<p>The purpose of Bazel tests is to confirm some property of the source files
checked into the repository. (In this document, "source files" includes test data,
golden outputs, and anything else kept under version control.) One
user writes a test to assert an invariant which they expect to be maintained.
Other users execute the test later to check whether the invariant has been
broken. If the test depends on any variables other than source files
(non-hermetic), its value is diminished, because the later users cannot be sure
their changes are at fault when the test stops passing.</p>
<p>Therefore the outcome of a test must depend only on:</p>
<ul>
<li>source files on which the test has a declared dependency</li>
<li>products of the build system on which the test has a declared dependency</li>
<li>resources whose behavior is guaranteed by the test runner to remain constant</li>
</ul>
<p>Currently, such behavior is not enforced. However, test runners reserve the
right to add such enforcement in the future.</p>
<h2>Role of the build system</h2>
<p>Test rules are analogous to binary rules in that each must yield an
executable program. For some languages, this is a stub program which combines
a language-specific harness with the test code. Test rules must produce other
outputs as well. In addition to the primary test executable, the test runner
will need a manifest of <b>runfiles</b>, input files which should be made
available to the test at runtime, and it may need information about the type,
size, and tags of a test.</p>
<p>The build system may use the runfiles to deliver code as well as data. (This
might be used as an optimization to make each test binary smaller by sharing
files across tests, e.g. through the use of dynamic linking.) The build system
should ensure that the generated executable loads these files via the runfiles
image provided by the test runner, rather than hardcoded references to absolute
locations in the source or output tree.</p>
<h2>Role of the test runner</h2>
<p>From the point of view of the test runner, each test is a program which can
be invoked with <code>execve()</code>. There may be other ways to execute
tests; for example, an IDE might allow the execution of Java tests in-process.
However, the result of running the test as a standalone process must be
considered authoritative. If a test process runs to completion and terminates
normally with an exit code of zero, the test has passed. Any other result is
considered a test failure. In particular, writing any of the strings
<code>PASS</code> or <code>FAIL</code> to stdout has no significance to the test
runner.</p>
<p>If a test takes too long to execute, exceeds some resource limit, or the test
runner otherwise detects prohibited behavior, it may choose to kill the test
and treat the run as a failure. The runner must not report the test as passing
after sending a signal to the test process or any children thereof.</p>
<p id="timeout">The whole test target (not individual methods or tests) is given a
limited amount of time to run to completion. The time limit for a test is based
on its <a href="be/common-definitions.html#test.timeout"><code>timeout</code></a>
attribute according to the following table:</p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th><a href="be/common-definitions.html#test.timeout"><code>timeout</code></a></th>
<th>Time Limit (sec.)</th>
</tr>
</thead>
<tbody>
<tr><td><code>short</code></td><td>60</td></tr>
<tr><td><code>moderate</code></td><td>300</td></tr>
<tr><td><code>long</code></td><td>900</td></tr>
<tr><td><code>eternal</code></td><td>3600</td></tr>
</tbody>
</table>
<p id="size">Tests which do not explicitly specify a timeout have one implied based on the
test's <a href="be/common-definitions.html#test.size"><code>size</code></a> as follows:</p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr>
<th><a href="be/common-definitions.html#test.size"><code>size</code></a></th>
<th>Implied timeout label</th>
</tr>
</thead>
<tbody>
<tr><td><code>small</code></td><td>short</td></tr>
<tr><td><code>medium</code></td><td>moderate</td></tr>
<tr><td><code>large</code></td><td>long</td></tr>
<tr><td><code>enormous</code></td><td>eternal</td></tr>
</tbody>
</table>
<p>For example a "large" test with no explicit timeout setting will be allotted
900 seconds to run. A "medium" test with a timeout of "short" will be allotted
60 seconds.</p>
<p>Unlike <code>timeout</code>, the <code>size</code> also determines the
limits for other resources like RAM, as described in
<a href="be/common-definitions.html#common-attributes-tests">Common
definitions</a>.</p>
<p>All combinations of <code>size</code> and <code>timeout</code> labels are
legal, so an "enormous" test may be declared to have a timeout of "short".
Presumably it would do some really horrible things very quickly.</p>
<p>Tests may return arbitrarily fast regardless of timeout. A test is not
penalized for an overgenerous timeout, although a warning may be issued: you
should generally set your timeout as tight as you can without incurring any
flakiness.</p>
<p>The test timeout can be overridden with the <code>--test_timeout</code>
bazel flag, e.g. for manually running under conditions which are known to be
slow. The <code>--test_timeout</code> values are in seconds. For example,
<code>--test_timeout=120</code> will set the test timeout to two minutes.</p>
<p>There is also a recommended lower bound for test timeouts as follows: </p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr><th>timeout</th><th>Time minimum (sec.)</th></tr>
</thead>
<tbody>
<tr><td><code>short</code></td><td>0</td></tr>
<tr><td><code>moderate</code></td><td>30</td></tr>
<tr><td><code>long</code></td><td>300</td></tr>
<tr><td><code>eternal</code></td><td>900</td></tr>
</tbody>
</table>
<p>For example, if a "moderate" test completes in 5.5s, consider setting
<code>timeout = "short"</code> or <code>size = "small"</code>. Using the bazel
<code>--test_verbose_timeout_warnings</code> command line option will show the
tests whose specified size is too big.</p>
<p>Test sizes and timeouts are specified in the BUILD file according to the specification
<a href="be/common-definitions.html#common-attributes-tests">here</a>.
If unspecified, a test's size will default to "medium".</p>
<p>If the main process of a test exits, but some of its children are still
running, the test runner should consider the run complete and count it as a
success or failure based on the exit code observed from the main process. The
test runner may kill any stray processes. Tests should not leak processes in
this fashion.</p>
<h2 id="test-sharding">Test sharding</h2>
<p>Tests can be parallelized via test sharding. See
<a href="user-manual.html#sharding-strategy"><code>--test_sharding_strategy</code></a>
and
<a href="be/common-definitions.html#common-attributes-tests"><code>shard_count</code></a>
to enable test sharding. When sharding is enabled, the test runner is launched
once per shard. The environment variable
<a href="#initial-conditions"><code>TEST_TOTAL_SHARDS</code></a> is the number
of shards, and
<a href="#initial-conditions"><code>TEST_SHARD_INDEX</code></a> is the shard
index, beginning at 0. Runners use this information to select which tests to
run - for example, using a round-robin strategy. Not all test runners support sharding.
If a runner supports sharding, it must create or update the last modified date
of the file specified by
<a href="#initial-conditions"><code>TEST_SHARD_STATUS_FILE</code></a>.
Otherwise, Bazel assumes it does not support sharding and will not launch
additional runners.</p>
<h2 id="initial-conditions">Initial conditions</h2>
<p>When executing a test, the test runner must establish certain initial
conditions.</p>
<p>The test runner must invoke each test with the path to the test
executable in <code>argv[0]</code>. This path must be relative and
beneath the test's current directory (which is in the runfiles tree,
see below).
The test runner should not pass any other arguments to a
test unless the user explicitly requests it.</p>
<p>The initial environment block shall be composed as follows:</p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr><th>Variable</th><th>Value</th><th>Status</th></tr>
</thead>
<tbody>
<tr><td><code>HOME</code></td><td>value of <code>$TEST_TMPDIR</code></td><td>recommended</td></tr>
<tr><td><code>LANG</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LANGUAGE</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_ALL</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_COLLATE</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_CTYPE</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_MESSAGES</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_MONETARY</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_NUMERIC</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LC_TIME</code></td><td><i>unset</i></td><td>required</td></tr>
<tr><td><code>LD_LIBRARY_PATH</code></td><td>colon-separated list of directories containing shared libraries</td><td>optional</td></tr>
<tr><td><code>JAVA_RUNFILES</code></td><td>value of <code>$TEST_SRCDIR</code></td><td>deprecated</td></tr>
<tr><td><code>LOGNAME</code></td><td>value of <code>$USER</code></td><td>required</td></tr>
<tr><td><code>PATH</code></td><td><code>/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.</code></td><td>recommended</td></tr>
<tr><td><code>PWD</code></td><td><code>$TEST_SRCDIR/<i>workspace-name</i></code></td><td>recommended</td></tr>
<tr><td><code>SHLVL</code></td><td><code>2</code></td><td>recommended</td></tr>
<tr><td><code>TEST_INFRASTRUCTURE_FAILURE_FILE</code></td><td>
absolute path to a private file in a writable directory (This file should only
be used to report failures originating from the testing infrastructure, not as a
general mechanism for reporting flaky failures of tests. In this context,
testing infrastructure is defined as systems or libraries that are not
test-specific, but can cause test failures by malfunctioning. The first line is
the name of the testing infrastructure component that caused the failure, the
second one a human-readable description of the failure. Additional lines are
ignored.)</td><td>optional</td></tr>
<tr><td><code>TEST_LOGSPLITTER_OUTPUT_FILE</code></td><td>absolute path to a private file in a writable directory (used to write Logsplitter protobuffer log)</td><td>optional</td></tr>
<tr><td><code>TEST_PREMATURE_EXIT_FILE</code></td><td>absolute path to a private file in a writable directory (used for catching calls to exit())</td><td>optional</td></tr>
<tr>
<td><code>TEST_RANDOM_SEED</code></td>
<td>If the <code class='flag'>--runs_per_test</code> option is used, TEST_RANDOM_SEED
is set to the <var>run number</var> (starting with 1) for each individual test run.</td>
<td>optional</td>
</tr>
<tr><td><code>TEST_TARGET</code></td><td>The name of the target being tested</td><td>optional</td></tr>
<tr><td><code>TEST_SIZE</code></td><td>The test <a href="#size"><code>size</code></a></td><td>optional</td></tr>
<tr><td><code>TEST_TIMEOUT</code></td><td>The test <a href="#timeout"><code>timeout</code></a> in seconds</td><td>optional</td></tr>
<tr><td><code>TEST_SHARD_INDEX</code></td><td>shard index, if <a href="#test-sharding">sharding</a> is used</td><td>optional</td></tr>
<tr><td><code>TEST_SHARD_STATUS_FILE</code></td><td>path to file to touch to indicate support for <a href="#test-sharding">sharding</a></td><td>optional</td></tr>
<tr><td><code>TEST_SRCDIR</code></td><td>absolute path to the base of the runfiles tree</td><td>required</td></tr>
<tr><td><code>TEST_TOTAL_SHARDS</code></td><td>total <a href="be/common-definitions.html#test.shard_count">shard count</a>, if <a href="#test-sharding">sharding</a> is used</td><td>optional</td></tr>
<tr><td><code>TEST_TMPDIR</code></td><td>absolute path to a private writable directory</td><td>required</td></tr>
<tr><td><code>TEST_WORKSPACE</code></td><td>the local repository's workspace name</td><td>optional</td></tr>
<tr><td><code>TEST_UNDECLARED_OUTPUTS_DIR</code></td><td>absolute path to a private writable directory (used to write undeclared test outputs)</td><td>optional</td></tr>
<tr><td><code>TEST_UNDECLARED_OUTPUTS_ANNOTATIONS_DIR</code></td><td>absolute path to a private writable directory (used to write undeclared test output annotation .part files).
</td><td>optional</td></tr>
<tr><td><code>TEST_WARNINGS_OUTPUT_FILE</code></td><td>absolute path to a private file in a writable directory (used to write test target warnings)</td><td>optional</td></tr>
<tr><td><code>TESTBRIDGE_TEST_ONLY</code></td><td>value of <a href="user-manual.html#flag--test_filter"><code>--test_filter</code></a>, if specified</td><td>optional</td></tr>
<tr><td><code>TZ</code></td><td><code>UTC</code></td><td>required</td></tr>
<tr><td><code>USER</code></td><td>value of <code>getpwuid(getuid())->pw_name</code></td><td>required</td></tr>
<tr>
<td><code>XML_OUTPUT_FILE</code></td>
<td>Location of the test result XML output file.
The XML schema is based on the
<a href="https://windyroad.com.au/dl/Open%20Source/JUnit.xsd">JUnit test result
schema</a>.
</td>
<td>optional</td>
</tr>
</tbody>
</table>
<br>
<p>The environment may contain additional entries. Tests should not depend on the
presence, absence, or value of any environment variable not listed above.</p>
<p>The initial working directory shall be <code>$TEST_SRCDIR/$TEST_WORKSPACE</code>.</p>
<p> The current process id, process group id, session id, and parent process
id are unspecified. The process may or may not be a process group leader or a
session leader. The process may or may not have a controlling terminal. The
process may have zero or more running or unreaped child processes. The process
should not have multiple threads when the test code gains control.</p>
<p>File descriptor 0 (stdin) shall be open for reading, but what it is attached
to is unspecified. Tests must not read from it. File descriptors 1 (stdout)
and 2 (stderr) shall be open for writing, but what they are attached to is
unspecified. It could be a terminal, a pipe, a regular file, or anything else
to which characters can be written. They may share an entry in the open file
table (meaning that they cannot seek independently). Tests should not inherit
any other open file descriptors.</p>
<p>The initial umask shall be 022 or 027.</p>
<p>No alarm or interval timer shall be pending.</p>
<p>The initial mask of blocked signals shall be empty. All signals shall be set
to their default action.</p>
<p>The initial resource limits, both soft and hard, should be set as follows:</p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr><th>Resource</th><th>Limit</th></tr>
</thead>
<tbody>
<tr><td>RLIMIT_AS</td><td>unlimited</td></tr>
<tr><td>RLIMIT_CORE</td><td>unspecified</td></tr>
<tr><td>RLIMIT_CPU</td><td>unlimited</td></tr>
<tr><td>RLIMIT_DATA</td><td>unlimited</td></tr>
<tr><td>RLIMIT_FSIZE</td><td>unlimited</td></tr>
<tr><td>RLIMIT_LOCKS</td><td>unlimited</td></tr>
<tr><td>RLIMIT_MEMLOCK</td><td>unlimited</td></tr>
<tr><td>RLIMIT_MSGQUEUE</td><td>unspecified</td></tr>
<tr><td>RLIMIT_NICE</td><td>unspecified</td></tr>
<tr><td>RLIMIT_NOFILE</td><td>at least 1024</td></tr>
<tr><td>RLIMIT_NPROC</td><td>unspecified</td></tr>
<tr><td>RLIMIT_RSS</td><td>unlimited</td></tr>
<tr><td>RLIMIT_RTPRIO</td><td>unspecified</td></tr>
<tr><td>RLIMIT_SIGPENDING</td><td>unspecified</td></tr>
<tr><td>RLIMIT_STACK</td><td>unlimited, or 2044KB <= rlim <= 8192KB</td></tr>
</tbody>
</table>
<p>The initial process times (as returned by <code>times()</code>) and resource
utilization (as returned by <code>getrusage()</code>) are unspecified.</p>
<p>The initial scheduling policy and priority are unspecified.</p>
<h2>Role of the host system</h2>
<p>In addition to the aspects of user context under direct control of the
test runner, the operating system on which tests execute must satisfy certain
properties for a test run to be valid.</p>
<h4>Filesystem</h4>
<p>
The root directory observed by a test may or may not be the real root directory.<br>
<code>/proc</code> shall be mounted.<br>
All build tools shall be present at the absolute paths under <code>/usr</code> used by a local installation.<br>
Paths starting with <code>/home</code> may not be available. Tests should not access any such paths.<br>
<code>/tmp</code>
shall be writable, but tests should avoid using these paths.<br>
Tests must not assume that any constant path is available for their exclusive use.<br>
Tests must not assume that atimes are enabled for any mounted filesystem.<br>
</p>
<h4>Users and groups</h4>
<p>The users root, nobody, and unittest must exist. The groups root, nobody,
and eng must exist.</p>
<p>Tests must be executed as a non-root user. The real and effective user ids
must be equal; likewise for group ids. Beyond this, the current user id, group
id, user name, and group name are unspecified. The set of supplementary group
ids is unspecified.</p>
<p>The current user id and group id must have corresponding names which can be
retrieved with <code>getpwuid()</code> and <code>getgrgid()</code>. The same
may not be true for supplementary group ids.</p>
<p>The current user must have a home directory. It may not be writable. Tests
must not attempt to write to it.</p>
<h4>Networking</h4>
<p>The hostname is unspecified. It may or may not contain a dot. Resolving
the hostname must give an IP address of the current host. Resolving the
hostname cut after the first dot must also work. The hostname localhost must
resolve.</p>
<h4>Other resources</h4>
<p>Tests are granted at least one CPU core. Others may be available but this
is not guaranteed. Other performance aspects of this core are not specified.
You can increase the reservation to a higher number of CPU cores by adding the
tag "cpu:n" (where n is a positive number) to a test rule. If a machine has
less total CPU cores than requested, Bazel will still run the test. If a test
uses <a href="#test-sharding">sharding</a>, each individual shard will reserve
the number of CPU cores specified here.</p>
<p>Tests may create subprocesses, but not process groups or sessions.</p>
<p>There is a limit on the number of input files a test may consume. This limit
is subject to change, but is currently in the range of tens of thousands of inputs.
</p>
<h4>Time and date</h4>
<p>The current time and date are unspecified. The system timezone is unspecified.
</p>
<p>X Windows may or may not be available. Tests that need an X server should
start Xvfb.</p>
<h2>Test interaction with the filesystem</h2>
<p>All file paths specified in test environment variables point to
somewhere on the local filesystem, unless otherwise specified.</p>
<p>
Tests should create files only within the directories specified by
<code>$TEST_TMPDIR</code> and <code>$TEST_UNDECLARED_OUTPUTS_DIR</code>
(if set).<br>
These directories will be initially empty.<br>
Tests must not attempt to remove, chmod, or otherwise alter these directories.<br>
These directories may be a symbolic links.<br>
The filesystem type of <code>$TEST_TMPDIR/.</code> remains unspecified.<br>
Tests may also write .part files to the <code>$TEST_UNDECLARED_OUTPUTS_ANNOTATIONS_DIR</code>
to annotate undeclared output files.</p>
<p>In rare cases, a test may be forced to create files in <code>/tmp</code>. For
example, <a href="https://serverfault.com/questions/641347">path length limits
for Unix domain sockets</a> typically require creating the socket under
<code>/tmp</code>. Bazel will be unable to track such files; the test itself must
take care to be hermetic, to use unique paths to avoid colliding with other,
simultaneously running tests and non-test processes, and to clean up the files it
creates in <code>/tmp</code>.</p>
<p>Tests must access inputs through the <b>runfiles</b> mechanism, or other
parts of the execution environment which are specifically intended to make
input files available.
Tests must not access other outputs of the
build system at paths inferred from the location of their own executable.</p>
<p>It is unspecified whether the runfiles tree contains regular files, symbolic
links, or a mixture. The runfiles tree may contain symlinks to directories.
Tests should avoid using paths containing <code>..</code> components within the
runfiles tree.</p>
<p>No directory, file, or symlink within the runfiles tree (including paths
which traverse symlinks) should be writable. (It follows that the initial
working directory should not be writable.) Tests must not assume that any part
of the runfiles is writable, or owned by the current user (i.e. chmod and chgrp
may fail).</p>
<p>The runfiles tree (including paths which traverse symlinks) must not change
during test execution. Parent directories and filesystem mounts must not change
in any way which affects the result of resolving a path within the runfiles
tree.</p>
<p>In order to catch early exit, a test may create a file at the path specified by
<code>TEST_PREMATURE_EXIT_FILE</code> upon start and remove it upon exit. If
Bazel sees the file when the test finishes, it will assume that the test exited
prematurely and mark it as having failed.</p>
<h2 id="tag-conventions">Tag conventions</h2>
<p>
Some tags in the test rules have a special
meaning. See also the <a href="be/common-definitions.html#common.tags">Bazel
Build Encyclopedia on the <code>tags</code> attribute</a>.
</p>
<table class="table table-bordered table-striped table-condensed">
<thead>
<tr><th>Tag</th><th>Meaning</th></tr>
</thead>
<tbody>
<tr>
<th><code>exclusive</code></th>
<td>run no other test at the same time</td>
</tr>
<tr>
<th><code>external</code></th>
<td>test has an external dependency; disable test caching</td>
</tr>
<tr>
<th><code>large</code></th>
<td><code>test_suite</code> convention; suite of large tests</td>
</tr>
<tr>
<th><code>manual</code>*</th>
<td>don't include test target in wildcard target patterns like <code>:...</code>, <code>:*</code>, or <code>:all</code>)</td>
</tr>
<tr>
<th><code>medium</code></th>
<td><code>test_suite</code> convention; suite of medium tests</td>
</tr>
<tr>
<th><code>small</code></th>
<td><code>test_suite</code> convention; suite of small tests</td>
</tr>
<tr>
<th><code>smoke</code></th>
<td>
<code>test_suite</code> convention; means it should be run before committing code changes
into the version control system
</td>
</tr>
</tbody>
</table>
*Note: bazel <code>query</code> does not respect the manual tag.
<h2>Runfiles</h2>
<p>In the following, assume there is a *_binary() rule labeled <code>//foo/bar:unittest</code>,
with a run-time dependency on the rule labeled <code>//deps/server:server</code>.</p>
<h4>Location</h4>
<p>The runfiles directory for a target <code>//foo/bar:unittest</code> is the directory
<code>$(WORKSPACE)/$(BINDIR)/foo/bar/unittest.runfiles</code>. This path is referred to as the
<code>runfiles_dir</code>.</p>
<h4>Dependencies</h4>
<p>The runfiles directory is declared as a compile-time dependency of the *_binary() rule.
The runfiles directory itself depends on the set of BUILD files that affect the *_binary() rule
or any of its compile-time or run-time dependencies. Modifying source files does not affect the
structure of the runfiles directory, and thus does not trigger any rebuilding.</p>
<h4>Contents</h4>
<p>The runfiles directory contains the following:</p>
<ul>
<li><b>Symlinks to run-time dependencies</b>: each OutputFile and CommandRule that is a run-time
dependency of the *_binary() rule is represented by one symlink in the runfiles directory.
The name of the symlink is <code>$(WORKSPACE)/package_name/rule_name</code>. For example, the
symlink for server would be named <code>$(WORKSPACE)/deps/server/server</code>, and the full path
would be <code>$(WORKSPACE)/foo/bar/unittest.runfiles/$(WORKSPACE)/deps/server/server</code>.
The destination of the symlink is the OutputFileName() of the OutputFile or CommandRule,
expressed as an absolute path. Thus, the destination of the symlink might be
<code>$(WORKSPACE)/linux-dbg/deps/server/42/server</code>.</li>
<li><b>Symlinks to sub-runfiles</b>: for every *_binary() Z that is a run-time depdendency of
*_binary() C, there is a second link in the runfiles directory of C to the runfiles of Z.
The name of the symlink is <code>$(WORKSPACE)/package_name/rule_name.runfiles</code>.
The target of the symlink is the runfiles directory. I.e. all subprograms share a common
runfiles directory.</li>
</ul>
|