1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
RayPlatform is a development framework to ease the creation of
massively distributed high performance computing applications.
RayPlatform is free software distributed under the terms of the GNU
Lesser General Public License, version 3 (LGPLv3).
Content creation is done by creating plugins that can be
added on the RayPlatform compute engine.
It uses the message-passing interface for interprocess communication, but this is
transparent to the developer.
## Models
Mainly two models are available:
- Rank model (with RayPlatform Plugin API)
- Actor Model (with the RayPlatform Actor Playground API)
Also: RayPlatform implements a "mini-ranks" programming model. It is a hybrid
programming model where MPI ranks and mini-ranks exist. Mini-ranks run in separate
threads inside a MPI rank process. This eases the usage of a superior hybrid
model with MPI and threads (pthread). But the actor model is superior in every aspect.
## Illustration of the RayPlatform Plugin API
```
+--------------------------------------------------------------------------+
| |
| Application |
| |
+------------------------+------------------------+------------------------+
| | | |
| Plugin | Plugin | Plugin |
| | | |
+---------+---------+ +---------+---------+ +---------+---------+ +
| | | | | | | | | |
| Adapter | Adapter | | Adapter | Adapter | | Adapter | Adapter | |
| | | | | | | | | |
+---------+---------+----+---------+---------+----+---------+---------+----+
| |
| RayPlatform |
| |
+--------------------------------------------------------------------------+
| |
| Message Passing Interface |
| |
+--------------------------------------------------------------------------+
```
## Projects using RayPlatform
- The Ray genome assembler (and also Ray Meta, Ray Communities, Ray Surveyor)
http://github.com/sebhtml/ray
- RayPlatform example
http://github.com/sebhtml/RayPlatform-example
- MessageWarden
http://github.com/sebhtml/MessageWarden
## Description
There is some documentation in Documentation/
You can also generate doxygen documentation.
The framework provides facilities for parallel software architecture,
communication, memory management, profiling, thread pools
and some structures.
It can be compiled as libRayPlatform.a, libRayPlatform.so or libRayPlatform.dll
using GNU Make or CMake.
### Parallel software architecture:
- a compute core implementation in which the main loop lives (core/ComputeCore.h);
- a system of plugins that can be registered with the ComputeCore (core/CorePlugin.h);
- a system of callbacks for message tags (handlers/MessageTagHandler.h);
- a system of callbacks for master modes (handlers/MasterModeHandler.h);
- a system of callbacks for slave modes (handlers/SlaveModeHandler.h);
- a system for master switches (scheduling/Switchman.h)
- a system for slave switches (scheduling/Switchman.h)
### Communication:
- a message inbox and outbox (structures/StaticVector.h);
- a virtual communicator for automated message aggregation (communication/VirtualCommunicator.h);
- a buffered data object for less-automated message aggregation (communication/BufferedData.h);
- a message router (including various graphs such as de Bruijn)
for jobs running on numerous cores (> 1000) (communication/MessageRouter.h and routing/*);
- a wrapper on top of the provided MPI library (communication/MessagesHandler.h).
### Memory management:
- a ring allocator (memory/RingAllocator.h);
- a "chunk" allocator (memory/MyAllocator.h);
- an allocator with real-time defragmentation (memory/ChunkAllocatorWithDefragmentation.h).
### Profiling:
- a profiler that reports granularity (profiling/Profiler.h);
- a tick counter for slave and master modes (profiling/TickLogger.h);
### Thread pools:
- a virtual processor containing a lot of workers (scheduling/VirtualProcessor.h);
- a general interface to define a worker (scheduling/Worker.h);
- a general way to use the virtual processor (scheduling/TaskCreator.h).
### Structures:
- very space-efficient sparse hash table (structures/MyHashTable.h);
- a splay tree (structures/SplayTree.h).
|
