1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285
|
cmake_minimum_required(VERSION 3.30)
include(RunCMake)
function(instrument test)
# Set Paths Variables
set(config "${CMAKE_CURRENT_LIST_DIR}/config")
set(ENV{CMAKE_CONFIG_DIR} ${config})
set(OPTIONS
"BUILD"
"BUILD_MAKE_PROGRAM"
"INSTALL"
"INSTALL_PARALLEL"
"TEST"
"NO_WARN"
"COPY_QUERIES"
"COPY_QUERIES_GENERATED"
"STATIC_QUERY"
"DYNAMIC_QUERY"
"TRACE_QUERY"
"MANUAL_HOOK"
"PRESERVE_DATA"
"NO_CONFIGURE"
"FAIL"
)
cmake_parse_arguments(ARGS "${OPTIONS}" "CHECK_SCRIPT;CONFIGURE_ARG" "" ${ARGN})
set(RunCMake_TEST_BINARY_DIR ${RunCMake_BINARY_DIR}/${test})
set(uuid "ec7aa2dc-b87f-45a3-8022-fe01c5f59984")
set(v1 ${RunCMake_TEST_BINARY_DIR}/.cmake/instrumentation-${uuid}/v1)
set(v1 ${v1} PARENT_SCOPE)
set(query_dir ${CMAKE_CURRENT_LIST_DIR}/query)
# Clear previous instrumentation data
# We can't use RunCMake_TEST_NO_CLEAN 0 because we preserve queries placed in the build tree after
if (ARGS_PRESERVE_DATA)
file(REMOVE_RECURSE ${RunCMake_TEST_BINARY_DIR}/CMakeFiles)
else()
file(REMOVE_RECURSE ${RunCMake_TEST_BINARY_DIR})
endif()
# Set hook command
set(static_query_hook_arg 0)
if (ARGS_STATIC_QUERY)
set(static_query_hook_arg 1)
endif()
set(trace_query_hook_arg 0)
if (ARGS_TRACE_QUERY)
set(trace_query_hook_arg 1)
endif()
set(GET_HOOK "\\\"${CMAKE_COMMAND}\\\" -P \\\"${RunCMake_SOURCE_DIR}/hook.cmake\\\" ${static_query_hook_arg} ${trace_query_hook_arg}")
# Load query JSON and cmake (with cmake_instrumentation(...)) files
set(query ${query_dir}/${test}.json.in)
set(cmake_file ${query_dir}/${test}.cmake)
if (EXISTS ${query})
file(MAKE_DIRECTORY ${v1}/query)
configure_file(${query} ${v1}/query/${test}.json)
elseif (EXISTS ${cmake_file})
list(APPEND ARGS_CONFIGURE_ARG "-DINSTRUMENT_COMMAND_FILE=${cmake_file}")
endif()
set(copy_loc ${RunCMake_TEST_BINARY_DIR}/query)
if (ARGS_COPY_QUERIES_GENERATED)
set(ARGS_COPY_QUERIES TRUE)
set(copy_loc ${v1}/query/generated) # Copied files here should be cleared on configure
endif()
if (ARGS_COPY_QUERIES)
set(CMAKE_COMMAND_QUOTE ${CMAKE_COMMAND})
if (CMAKE_COMMAND MATCHES " ")
set(CMAKE_COMMAND_QUOTE "\\\"${CMAKE_COMMAND}\\\"")
endif()
file(MAKE_DIRECTORY ${copy_loc})
set(generated_queries "0;1;2")
foreach(n IN LISTS generated_queries)
configure_file(
"${query_dir}/generated/query-${n}.json.in"
"${copy_loc}/query-${n}.json"
)
endforeach()
endif()
# Configure Test Case
set(RunCMake_TEST_NO_CLEAN 1)
if (ARGS_NO_WARN)
list(APPEND ARGS_CONFIGURE_ARG "-Wno-dev")
endif()
if (ARGS_FAIL)
list(APPEND ARGS_CONFIGURE_ARG "-DFAIL=ON")
endif()
set(RunCMake_TEST_SOURCE_DIR ${RunCMake_SOURCE_DIR}/project)
if(NOT RunCMake_GENERATOR_IS_MULTI_CONFIG)
set(maybe_CMAKE_BUILD_TYPE -DCMAKE_BUILD_TYPE=Debug)
endif()
if (NOT ARGS_NO_CONFIGURE)
run_cmake_with_options(${test} ${ARGS_CONFIGURE_ARG} ${maybe_CMAKE_BUILD_TYPE})
endif()
# Follow-up Commands
if (ARGS_BUILD)
set(cmake_build_args --config Debug)
set(additional_build_args)
if (ARGS_FAIL)
# Tests with ARGS_FAIL expect all targets to build, including the ones
# which should succeed and those which should fail.
if (RunCMake_GENERATOR MATCHES "Ninja")
set(keep_going_arg -k 0)
elseif (RunCMake_GENERATOR MATCHES "FASTBuild")
set(keep_going_arg -nostoponerror)
else()
set(keep_going_arg -k)
endif()
string(APPEND additional_build_args ${keep_going_arg})
# Merge stdout and stderr because different compilers will throw their
# errors to different places.
set(RunCMake_TEST_OUTPUT_MERGE 1)
endif()
run_cmake_command(${test}-build
${CMAKE_COMMAND} --build . ${cmake_build_args} -- ${additional_build_args}
)
if (ARGS_FAIL)
unset(RunCMake_TEST_OUTPUT_MERGE)
endif()
endif()
if (ARGS_BUILD_MAKE_PROGRAM)
set(RunCMake_TEST_OUTPUT_MERGE 1)
# Force reconfigure to test for double preBuild & postBuild hooks
file(TOUCH ${RunCMake_TEST_BINARY_DIR}/CMakeCache.txt)
run_cmake_command(${test}-make-program ${RunCMake_MAKE_PROGRAM})
unset(RunCMake_TEST_OUTPUT_MERGE)
endif()
if (ARGS_INSTALL)
run_cmake_command(${test}-install ${CMAKE_COMMAND} --install . --prefix install --config Debug)
endif()
if (ARGS_TEST)
run_cmake_command(${test}-test ${CMAKE_CTEST_COMMAND} . -C Debug)
endif()
if (ARGS_MANUAL_HOOK)
run_cmake_command(${test}-index ${CMAKE_CTEST_COMMAND} --collect-instrumentation .)
endif()
# Run Post-Test Checks
# Check scripts need to run after ALL run_cmake_command have finished
if (ARGS_CHECK_SCRIPT)
set(RunCMake-check-file ${ARGS_CHECK_SCRIPT})
set(RunCMake_CHECK_ONLY 1)
run_cmake(${test}-verify)
unset(RunCMake-check-file)
unset(RunCMake_CHECK_ONLY)
endif()
endfunction()
# Bad Queries
instrument(bad-option)
instrument(bad-hook)
instrument(empty)
instrument(bad-version)
# Verify Hooks Run and Index File
instrument(hooks-1 BUILD INSTALL TEST STATIC_QUERY)
instrument(hooks-2 BUILD INSTALL TEST)
instrument(hooks-no-callbacks MANUAL_HOOK)
# Check data file contents for optional query data
instrument(no-query
BUILD INSTALL TEST
CHECK_SCRIPT check-data-dir.cmake
)
instrument(dynamic-query
BUILD INSTALL TEST DYNAMIC_QUERY
CHECK_SCRIPT check-data-dir.cmake
)
instrument(both-query
BUILD INSTALL TEST DYNAMIC_QUERY
CHECK_SCRIPT check-data-dir.cmake
)
# Test cmake_instrumentation command
instrument(cmake-command
COPY_QUERIES NO_WARN STATIC_QUERY DYNAMIC_QUERY
CHECK_SCRIPT check-generated-queries.cmake
)
instrument(cmake-command-data
COPY_QUERIES NO_WARN BUILD INSTALL TEST DYNAMIC_QUERY
CHECK_SCRIPT check-data-dir.cmake
)
instrument(cmake-command-bad-api-version NO_WARN)
instrument(cmake-command-bad-data-version NO_WARN)
instrument(cmake-command-missing-version NO_WARN)
instrument(cmake-command-bad-arg NO_WARN)
instrument(cmake-command-parallel-install
BUILD INSTALL TEST NO_WARN INSTALL_PARALLEL DYNAMIC_QUERY
CHECK_SCRIPT check-data-dir.cmake)
instrument(cmake-command-resets-generated
NO_WARN COPY_QUERIES_GENERATED
CHECK_SCRIPT check-data-dir.cmake
)
instrument(cmake-command-cmake-build
NO_WARN BUILD
CHECK_SCRIPT check-no-make-program-hooks.cmake
)
if(RunCMake_GENERATOR STREQUAL "Borland Makefiles")
# Borland 'make' has no '-k' flag.
set(Skip_COMMAND_FAILURES_Case 1)
endif()
if(NOT Skip_COMMAND_FAILURES_Case)
instrument(cmake-command-failures
FAIL NO_WARN BUILD TEST INSTALL
CHECK_SCRIPT check-data-dir.cmake
)
endif()
# Test CUSTOM_CONTENT
instrument(cmake-command-custom-content
NO_WARN BUILD
CONFIGURE_ARG "-DN=1"
)
instrument(cmake-command-custom-content
NO_WARN BUILD PRESERVE_DATA
CONFIGURE_ARG "-DN=2"
CHECK_SCRIPT check-custom-content.cmake
)
set(indexDir ${v1}/data/index)
set(fakeIndex ${indexDir}/index-0.json)
file(MAKE_DIRECTORY ${indexDir})
file(TOUCH ${fakeIndex})
# fakeIndex newer than all content files prevents their deletion
set(EXPECTED_CONTENT_FILES 2)
instrument(cmake-command-custom-content
NO_WARN NO_CONFIGURE MANUAL_HOOK PRESERVE_DATA
CHECK_SCRIPT check-custom-content-removed.cmake
)
file(REMOVE ${fakeIndex})
# old content files will be removed if no index file exists
set(EXPECTED_CONTENT_FILES 1)
instrument(cmake-command-custom-content
NO_WARN NO_CONFIGURE MANUAL_HOOK PRESERVE_DATA
CHECK_SCRIPT check-custom-content-removed.cmake
)
instrument(cmake-command-custom-content-bad-type NO_WARN)
instrument(cmake-command-custom-content-bad-content NO_WARN)
# Test Google trace
instrument(trace-query
BUILD INSTALL TEST TRACE_QUERY
CHECK_SCRIPT check-generated-queries.cmake
)
instrument(cmake-command-trace
NO_WARN BUILD INSTALL TEST TRACE_QUERY
)
instrument(cmake-command-trace
NO_WARN BUILD PRESERVE_DATA
CHECK_SCRIPT check-trace-removed.cmake
)
# Test make/ninja hooks
if(RunCMake_GENERATOR STREQUAL "FASTBuild")
# FIXME(#27184): This does not work for FASTBuild.
set(Skip_BUILD_MAKE_PROGRAM_Case 1)
elseif(RunCMake_GENERATOR STREQUAL "MSYS Makefiles")
# FIXME(#27079): This does not work for MSYS Makefiles.
set(Skip_BUILD_MAKE_PROGRAM_Case 1)
elseif(RunCMake_GENERATOR STREQUAL "NMake Makefiles")
execute_process(
COMMAND "${RunCMake_MAKE_PROGRAM}" -?
OUTPUT_VARIABLE nmake_out
ERROR_VARIABLE nmake_out
RESULT_VARIABLE nmake_res
OUTPUT_STRIP_TRAILING_WHITESPACE
)
if(nmake_res EQUAL 0 AND nmake_out MATCHES "Program Maintenance Utility[^\n]+Version ([1-9][0-9.]+)")
set(nmake_version "${CMAKE_MATCH_1}")
else()
message(FATAL_ERROR "'nmake -?' reported:\n${nmake_out}")
endif()
if(nmake_version VERSION_LESS 9)
set(Skip_BUILD_MAKE_PROGRAM_Case 1)
endif()
endif()
if(NOT Skip_BUILD_MAKE_PROGRAM_Case)
instrument(cmake-command-make-program
NO_WARN BUILD_MAKE_PROGRAM
CHECK_SCRIPT check-make-program-hooks.cmake)
instrument(cmake-command-build-snippet
NO_WARN BUILD_MAKE_PROGRAM
CHECK_SCRIPT check-data-dir.cmake)
endif()
|