1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402
|
#!/usr/bin/env bash
set -eET
flags=('--dummy-flag')
num_jobs=${BATS_NUMBER_OF_PARALLEL_JOBS:-1}
extended_syntax=''
BATS_ABORT_ON_FAILURE=
BATS_TRACE_LEVEL="${BATS_TRACE_LEVEL:-0}"
declare -r BATS_RETRY_RETURN_CODE=126
export BATS_TEST_RETRIES=0 # no retries by default
while [[ "$#" -ne 0 ]]; do
case "$1" in
-j)
shift
num_jobs="$1"
;;
-T)
flags+=('-T')
;;
-x)
flags+=('-x')
extended_syntax=1
;;
--abort)
BATS_ABORT_ON_FAILURE=1
;;
--no-parallelize-within-files)
# use singular to allow for users to override in file
BATS_NO_PARALLELIZE_WITHIN_FILE=1
;;
--dummy-flag) ;;
--trace)
flags+=('--trace')
;;
--print-output-on-failure)
flags+=(--print-output-on-failure)
;;
--show-output-of-passing-tests)
flags+=(--show-output-of-passing-tests)
;;
--verbose-run)
flags+=(--verbose-run)
;;
--gather-test-outputs-in)
shift
flags+=(--gather-test-outputs-in "$1")
;;
*)
break
;;
esac
shift
done
export BATS_TEST_FILE_NUMBER="$1"
filename="$2"
TESTS_FILE="$3"
if [[ ! -f "$filename" ]]; then
printf 'Testfile "%s" not found\n' "$filename" >&2
exit 1
fi
export BATS_TEST_FILENAME="$filename"
# shellcheck source=lib/bats-core/preprocessing.bash
# shellcheck disable=SC2153
source "$BATS_ROOT/$BATS_LIBDIR/bats-core/preprocessing.bash"
bats_run_setup_file() {
# shellcheck source=lib/bats-core/tracing.bash
# shellcheck disable=SC2153
source "$BATS_ROOT/$BATS_LIBDIR/bats-core/tracing.bash"
# shellcheck source=lib/bats-core/test_functions.bash
# shellcheck disable=SC2153
source "$BATS_ROOT/$BATS_LIBDIR/bats-core/test_functions.bash"
_bats_test_functions_setup -1 # invalid TEST_NUMBER, as this is not a test
exec 3<&1
# these are defined only to avoid errors when referencing undefined variables down the line
# shellcheck disable=2034
BATS_TEST_NAME= # used in tracing.bash
# shellcheck disable=2034
BATS_TEST_COMPLETED= # used in tracing.bash
BATS_SOURCE_FILE_COMPLETED=
BATS_SETUP_FILE_COMPLETED=
BATS_TEARDOWN_FILE_COMPLETED=
# shellcheck disable=2034
BATS_ERROR_STATUS= # used in tracing.bash
touch "$BATS_OUT"
bats_setup_tracing
trap 'bats_file_teardown_trap' EXIT
local status=0
# get the setup_file/teardown_file functions for this file (if it has them)
# shellcheck disable=SC1090
source "$BATS_TEST_SOURCE"
BATS_SOURCE_FILE_COMPLETED=1
bats_set_stacktrace_limit
setup_file >>"$BATS_OUT" 2>&1
BATS_SETUP_FILE_COMPLETED=1
}
bats_run_teardown_file() {
local bats_teardown_file_status=0
# avoid running the therdown trap due to errors in teardown_file
trap 'bats_file_exit_trap' EXIT
bats_set_stacktrace_limit
# rely on bats_error_trap to catch failures
teardown_file >>"$BATS_OUT" 2>&1 || bats_teardown_file_status=$?
if ((bats_teardown_file_status == 0)); then
BATS_TEARDOWN_FILE_COMPLETED=1
elif [[ -n "${BATS_SETUP_FILE_COMPLETED:-}" ]]; then
BATS_DEBUG_LAST_STACK_TRACE_IS_VALID=1
BATS_ERROR_STATUS=$bats_teardown_file_status
return $BATS_ERROR_STATUS
fi
}
# shellcheck disable=SC2317,SC2329
bats_file_teardown_trap() {
bats_run_teardown_file
bats_file_exit_trap in-teardown_trap
}
# shellcheck source=lib/bats-core/common.bash
source "$BATS_ROOT/$BATS_LIBDIR/bats-core/common.bash"
# shellcheck disable=SC2317,SC2329
bats_file_exit_trap() {
local -r last_return_code=$?
if [[ ${1:-} != in-teardown_trap ]]; then
BATS_ERROR_STATUS=$last_return_code
fi
trap - ERR EXIT
local failure_reason
local -i failure_test_index=$((BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE + 1))
if [[ -n "${BATS_TEST_SKIPPED-}" ]]; then
export BATS_TEST_SKIPPED # indicate to exec-test that it should skip
# print skip message for each test (use this to avoid reimplementing filtering)
bats_run_tests 1<&3 # restore original stdout (this is running in setup_file's redirection to BATS_OUT)
bats_exec_file_status=0 # this should not lead to errors
elif [[ -z "$BATS_SETUP_FILE_COMPLETED" || -z "$BATS_TEARDOWN_FILE_COMPLETED" ]]; then
if [[ -z "$BATS_SETUP_FILE_COMPLETED" ]]; then
failure_reason='setup_file'
elif [[ -z "$BATS_TEARDOWN_FILE_COMPLETED" ]]; then
failure_reason='teardown_file'
failure_test_index=$((BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE + ${#tests_to_run[@]} + 1))
elif [[ -z "$BATS_SOURCE_FILE_COMPLETED" ]]; then
failure_reason='source'
else
failure_reason='unknown internal'
fi
printf "not ok %d %s\n" "$failure_test_index" "$failure_reason failed" >&3
local stack_trace
bats_get_failure_stack_trace stack_trace
bats_print_stack_trace "${stack_trace[@]}" >&3
bats_print_failed_command "${stack_trace[@]}" >&3
bats_prefix_lines_for_tap_output <"$BATS_OUT" | bats_replace_filename >&3
rm -rf "$BATS_OUT"
bats_exec_file_status=1
fi
# setup_file not executed but defined in this test file? -> might be defined in the wrong file
if [[ -z "${BATS_SETUP_SUITE_COMPLETED-}" ]] && declare -F setup_suite >/dev/null; then
bats_generate_warning 3 --no-stacktrace "$BATS_TEST_FILENAME"
fi
exit "$bats_exec_file_status"
}
function setup_file() {
return 0
}
function teardown_file() {
return 0
}
bats_forward_output_of_parallel_test() {
local test_number_in_suite=$1
local status=0
wait "$(cat "$output_folder/$test_number_in_suite/pid")" || status=1
cat "$output_folder/$test_number_in_suite/stdout"
cat "$output_folder/$test_number_in_suite/stderr" >&2
return $status
}
bats_is_next_parallel_test_finished() {
local PID
# get the pid of the next potentially finished test
PID=$(cat "$output_folder/$((test_number_in_suite_of_last_finished_test + 1))/pid")
# try to send a signal to this process
# if it fails, the process exited,
# if it succeeds, the process is still running
if kill -0 "$PID" 2>/dev/null; then
return 1
fi
}
# prints output from all tests in the order they were started
# $1 == "blocking": wait for a test to finish before printing
# != "blocking": abort printing, when a test has not finished
bats_forward_output_for_parallel_tests() {
local status=0
# was the next test already started?
while ((test_number_in_suite_of_last_finished_test + 1 <= test_number_in_suite)); do
# if we are okay with waiting or if the test has already been finished
if [[ "$1" == "blocking" ]] || bats_is_next_parallel_test_finished; then
((++test_number_in_suite_of_last_finished_test))
bats_forward_output_of_parallel_test "$test_number_in_suite_of_last_finished_test" || status=$?
else
# non-blocking and the process has not finished -> abort the printing
break
fi
done
return $status
}
bats_run_test_with_retries() { # <args>
local status=0
local should_try_again=1 try_number
for ((try_number = 1; should_try_again; ++try_number)); do
if "$BATS_LIBEXEC/bats-exec-test" "$@" "$try_number"; then
should_try_again=0
else
status=$?
if ((status == BATS_RETRY_RETURN_CODE)); then
should_try_again=1
status=0 # this is not the last try -> reset status
else
should_try_again=0
bats_exec_file_status=$status
fi
fi
done
return $status
}
bats_run_tests_in_parallel() {
local output_folder="$BATS_RUN_TMPDIR/parallel_output"
local status=0
mkdir -p "$output_folder"
# shellcheck source=lib/bats-core/semaphore.bash
source "$BATS_ROOT/$BATS_LIBDIR/bats-core/semaphore.bash"
bats_semaphore_setup
# the test_number_in_file is not yet incremented -> one before the next test to run
local test_number_in_suite_of_last_finished_test="$BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE" # stores which test was printed last
local test_number_in_file=0 test_number_in_suite=$BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE
for test_name in "${tests_to_run[@]}"; do
# Only handle non-empty lines
if [[ $test_name ]]; then
((++test_number_in_suite))
((++test_number_in_file))
mkdir -p "$output_folder/$test_number_in_suite"
bats_semaphore_run "$output_folder/$test_number_in_suite" \
bats_run_test_with_retries "${flags[@]}" "$filename" "$test_name" "$test_number_in_suite" "$test_number_in_file" \
>"$output_folder/$test_number_in_suite/pid"
fi
# print results early to get interactive feedback
bats_forward_output_for_parallel_tests non-blocking || status=1 # ignore if we did not finish yet
if (( BATS_ABORT_ON_FAILURE && status >0 )); then
break
fi
done
bats_forward_output_for_parallel_tests blocking || status=1
return $status
}
bats_read_tests_list_file() {
local line_number=0
tests_to_run=()
# the global test number must be visible to traps -> not local
local test_number_in_suite=''
while read -r test_line; do
# check if the line begins with filename
# filename might contain some hard to parse characters,
# use simple string operations to work around that issue
if [[ "$filename" == "${test_line::${#filename}}" ]]; then
# get the rest of the line without the separator \t
test_name=${test_line:$((1 + ${#filename}))}
tests_to_run+=("$test_name")
# save the first test's number for later iteration
# this assumes that tests for a file are stored consecutive in the file!
if [[ -z "$test_number_in_suite" ]]; then
test_number_in_suite=$line_number
fi
fi
((++line_number))
done <"$TESTS_FILE"
BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE="$test_number_in_suite"
declare -ri BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE # mark readonly (cannot merge assignment, because value would be lost)
}
bats_run_tests() {
bats_exec_file_status=0
# TODO: this does not seem to be used anymore?
if [[ "${BATS_RUN_TESTS_SKIPPED-}" ]]; then
# shellcheck disable=SC2317,SC2329
bats_test_begin() {
printf "ok %d %s # skip %s\n" "$test_number_in_suite" "$1" "$BATS_RUN_TESTS_SKIPPED_REASON" >&3
return 1
}
local test_number_in_suite=0
for test_name in "${tests_to_run[@]}"; do
((++test_number_in_suite))
eval "$test_name" || true
done
exit 0
fi
if [[ "$num_jobs" -lt 1 ]]; then
printf 'Invalid number of jobs: %s\n' "$num_jobs" >&2
exit 1
fi
if [[ "$num_jobs" != 1 && "${BATS_NO_PARALLELIZE_WITHIN_FILE-False}" == False ]]; then
export BATS_SEMAPHORE_NUMBER_OF_SLOTS="$num_jobs"
bats_run_tests_in_parallel "$BATS_RUN_TMPDIR/parallel_output" || bats_exec_file_status=1
else
local test_number_in_suite=$BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE \
test_number_in_file=0
for test_name in "${tests_to_run[@]}"; do
#echo "bats-exec-file: execute test ${test_name}" >&2
if [[ "${BATS_INTERRUPTED-NOTSET}" != NOTSET ]]; then
bats_exec_file_status=130 # bash's code for SIGINT exits
break
fi
# Only handle non-empty lines
if [[ -n ${test_name-} ]]; then
((++test_number_in_suite))
((++test_number_in_file))
bats_run_test_with_retries "${flags[@]}" "$filename" "$test_name" \
"$test_number_in_suite" "$test_number_in_file" || bats_exec_file_status=$?
if (( BATS_ABORT_ON_FAILURE && bats_exec_file_status > 0 )); then
break
fi
fi
done
fi
}
bats_create_file_tempdirs() {
local bats_files_tmpdir="${BATS_RUN_TMPDIR}/file"
if ! mkdir -p "$bats_files_tmpdir"; then
printf 'Failed to create %s\n' "$bats_files_tmpdir" >&2
exit 1
fi
BATS_FILE_TMPDIR="$bats_files_tmpdir/${BATS_TEST_FILE_NUMBER?}"
if ! mkdir "$BATS_FILE_TMPDIR"; then
printf 'Failed to create BATS_FILE_TMPDIR=%s\n' "$BATS_FILE_TMPDIR" >&2
exit 1
fi
ln -s "$BATS_TEST_FILENAME" "$BATS_FILE_TMPDIR-$(basename "$BATS_TEST_FILENAME").source_file"
export BATS_FILE_TMPDIR
}
trap 'BATS_INTERRUPTED=true' INT
BATS_FILE_FIRST_TEST_NUMBER_IN_SUITE=0 # predeclare as Bash 3.2 does not support declare -g
bats_read_tests_list_file
# don't run potentially expensive setup/teardown_file
# when there are no tests to run
if [[ ${#tests_to_run[@]} -eq 0 ]]; then
exit 0
fi
if [[ -n "$extended_syntax" ]]; then
printf "suite %s\n" "$filename"
fi
# requires the test list to be read but not empty
bats_create_file_tempdirs
export BATS_OUT="${BATS_RUN_TMPDIR}/file/${BATS_TEST_FILE_NUMBER?}-${BATS_TEST_FILENAME##*/}.out"
bats_export_preprocess_source_BATS_TEST_SOURCE
trap bats_interrupt_trap INT
bats_run_setup_file
# during tests, we don't want to get backtraces from this level
# just wait for the test to be interrupted and display their trace
trap 'BATS_INTERRUPTED=true' INT
bats_run_tests
trap bats_interrupt_trap INT
bats_run_teardown_file
exit $bats_exec_file_status
|