1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234
|
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file LICENSE.rst or https://cmake.org/licensing for details. */
#pragma once
#include "cmConfigure.h" // IWYU pragma: keep
#include <cstddef>
#include <list>
#include <map>
#include <memory>
#include <set>
#include <string>
#include <vector>
#include <cm/optional>
#include <cm/string_view>
#include "cmCTest.h"
#include "cmCTestResourceAllocator.h"
#include "cmCTestResourceSpec.h"
#include "cmCTestTestHandler.h"
#include "cmUVHandlePtr.h"
#include "cmUVJobServerClient.h"
struct cmCTestBinPackerAllocation;
class cmCTestRunTest;
/** \class cmCTestMultiProcessHandler
* \brief run parallel ctest
*
* cmCTestMultiProcessHandler
*/
class cmCTestMultiProcessHandler
{
friend class TestComparator;
friend class cmCTestRunTest;
public:
struct TestSet : public std::set<int>
{
};
struct TestInfo
{
TestSet Depends;
};
struct TestMap : public std::map<int, TestInfo>
{
};
struct TestList : public std::vector<int>
{
};
struct PropertiesMap
: public std::map<int, cmCTestTestHandler::cmCTestTestProperties*>
{
};
struct ResourceAllocation
{
std::string Id;
unsigned int Slots;
};
cmCTestMultiProcessHandler(cmCTest* ctest, cmCTestTestHandler* handler);
virtual ~cmCTestMultiProcessHandler();
// Set the tests
bool SetTests(TestMap tests, PropertiesMap properties);
// Set the max number of tests that can be run at the same time.
void SetParallelLevel(cm::optional<size_t> level);
void SetTestLoad(unsigned long load);
virtual void RunTests();
void PrintOutputAsJson();
void PrintTestList();
void PrintLabels();
void SetPassFailVectors(std::vector<std::string>* passed,
std::vector<std::string>* failed)
{
this->Passed = passed;
this->Failed = failed;
}
void SetTestResults(std::vector<cmCTestTestHandler::cmCTestTestResult>* r)
{
this->TestResults = r;
}
cmCTestTestHandler* GetTestHandler() { return this->TestHandler; }
void SetRepeatMode(cmCTest::Repeat mode, int count)
{
this->RepeatMode = mode;
this->RepeatCount = count;
}
void SetResourceSpecFile(std::string const& resourceSpecFile)
{
this->ResourceSpecFile = resourceSpecFile;
}
void SetQuiet(bool b) { this->Quiet = b; }
void CheckResourceAvailability();
protected:
// Start the next test or tests as many as are allowed by
// ParallelLevel
void StartNextTests();
void StartTestProcess(int test);
void StartTest(int test);
// Mark the checkpoint for the given test
void WriteCheckpoint(int index);
void UpdateCostData();
void ReadCostData();
// Return index of a test based on its name
int SearchByName(cm::string_view name);
void CreateTestCostList();
void GetAllTestDependencies(int test, TestList& dependencies);
void CreateSerialTestCostList();
void CreateParallelTestCostList();
// Removes the checkpoint file
void MarkFinished();
void FinishTestProcess(std::unique_ptr<cmCTestRunTest> runner, bool started);
void StartNextTestsOnIdle();
void StartNextTestsOnTimer();
void RemoveTest(int index);
// Check if we need to resume an interrupted test set
void CheckResume();
// Check if there are any circular dependencies
bool CheckCycles();
int FindMaxIndex();
inline size_t GetProcessorsUsed(int index);
std::string GetName(int index);
bool CheckStopOnFailure();
bool CheckStopTimePassed();
void SetStopTimePassed();
void InitializeLoop();
void FinalizeLoop();
bool ResourceLocksAvailable(int test);
void LockResources(int index);
void UnlockResources(int index);
enum class ResourceAvailabilityError
{
NoResourceType,
InsufficientResources,
};
bool Complete();
bool AllocateResources(int index);
bool TryAllocateResources(
int index,
std::map<std::string, std::vector<cmCTestBinPackerAllocation>>&
allocations,
std::map<std::string, ResourceAvailabilityError>* errors = nullptr);
void DeallocateResources(int index);
bool AllResourcesAvailable();
bool InitResourceAllocator(std::string& error);
bool CheckGeneratedResourceSpec();
private:
cmCTest* CTest;
cmCTestTestHandler* TestHandler;
bool UseResourceSpec = false;
cmCTestResourceSpec ResourceSpec;
std::string ResourceSpecFile;
std::string ResourceSpecSetupFixture;
cm::optional<std::size_t> ResourceSpecSetupTest;
bool HasInvalidGeneratedResourceSpec = false;
// Tests pending selection to start. They may have dependencies.
TestMap PendingTests;
// List of pending test indexes, ordered by cost.
std::list<int> OrderedTests;
// Total number of tests we'll be running
size_t Total = 0;
// Number of tests that are complete
size_t Completed = 0;
size_t RunningCount = 0;
std::set<size_t> ProcessorsAvailable;
size_t HaveAffinity;
bool StopTimePassed = false;
// list of test properties (indices concurrent to the test map)
PropertiesMap Properties;
std::map<int, std::string> TestOutput;
std::vector<std::string>* Passed;
std::vector<std::string>* Failed;
std::vector<std::string> LastTestsFailed;
std::set<std::string> ProjectResourcesLocked;
std::map<int,
std::vector<std::map<std::string, std::vector<ResourceAllocation>>>>
AllocatedResources;
std::map<int, std::map<std::string, ResourceAvailabilityError>>
ResourceAvailabilityErrors;
cmCTestResourceAllocator ResourceAllocator;
std::vector<cmCTestTestHandler::cmCTestTestResult>* TestResults;
// Get the maximum number of processors that may be used at once.
size_t GetParallelLevel() const;
// With no '-j' option, default to serial testing.
cm::optional<size_t> ParallelLevel = 1;
// Fallback parallelism limit when '-j' is given with no value.
size_t ParallelLevelDefault;
// 'make' jobserver client. If connected, we acquire a token
// for each test before running its process.
cm::optional<cmUVJobServerClient> JobServerClient;
// List of tests that are queued to run when a token is available.
std::list<int> JobServerQueuedTests;
// Callback invoked when a token is received.
void JobServerReceivedToken();
unsigned long TestLoad = 0;
unsigned long FakeLoadForTesting = 0;
cm::uv_loop_ptr Loop;
cm::uv_idle_ptr StartNextTestsOnIdle_;
cm::uv_timer_ptr StartNextTestsOnTimer_;
bool HasCycles = false;
cmCTest::Repeat RepeatMode = cmCTest::Repeat::Never;
int RepeatCount = 1;
bool Quiet = false;
bool SerialTestRunning = false;
};
|