1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
|
Publishing presentation data from Synth core into the UI
--------------------------------------------------------
In winter 2023/24, it was established as a goal for further development to
get a better separation between Synth core and the User Interface and especially
to remove direct data access from the UI thread reaching out into the Synth internals.
For sending action commands and for retrieving individual state values, there was already
an existing and mature system in place, the `InterChange` -- allowing Synth and UI to
communicate asynchronously over a set of lock-free queues (actually implemented as ringbuffers
with atomic write and read position). However, this existing system leans towards high performance
data exchange, since it is also used for MIDI control commands, and thus has very limited bandwidth.
Yet obviously, the UI needs to retrieve extended information from the Synth core state, most notably
to display the various oscillator, transfer function and envelope graphs.
Thus a new communication scheme was established, based on a component **GuiDataExchange**
residing within the InterChange associated to each SynthEngine instance. While there is only a
single instance of the FLTK-UI, Yoshimi is capable of running several instances of the SynthEngine,
each performing within a separate thread and relying on a distinct InterChange instance. Only one
of theses Synth instances however, the »master« or `primarySynth`, loads global configuration
and can cause shutdown of the application.
Conceptual View
~~~~~~~~~~~~~~~
On an abstract level, GuiDataExchange establishes an unlimited number of _private data channels,_
allowing to push _arbitrary data_ from the Synth core into the UI in a thread-safe and asynchronous way.
NOTE: a prototype / demo setup of this system can be found in GuiDataExchangeTest.cpp,
which originally was used to draft and verify this new component.
Each of these individual "virtual data channels" is represented by a connection object, which in fact
is only a front-end handle. The connection is _templated_ to a specific data type an is only able to
transport data of this type; data to be published must be trivially destructible and copy/assignable.
We refer to such data records, which are used only to transport data from one subsystem to another
as »Data Transfer Object« (DTO).
Connections must be created from the GuiDataExchange and will automatically embed an opaque yet unique
connection identifier, plus another identifier to encode the data type that can be sent over the connection.
Actually, this bare bone ID of a connection is encapsulated as `GuiDataExchange::RoutingTag`. Given such
a tag, together with a reference to the managing GuiDataExchange, a connection can be (re-)established.
The Connection handle (object) is the primary interface used by client code to publish data.
It allows to `publish(DAT const&)` a copy of a given DTO into the virtual channel. On the other side
of this connection, there can be zero, one or several receivers, modelled by the interface `Subscription`.
Whenever a data object is _published_, each of the receivers (Subscription instances) will get a
_copy_ of this object delivered into an internal buffer, called the `MirrorData` block.
The idea is that some facility in the Core holds a connection handle, while the actual UI widgets
in need of the data will embed a MirrorData block, and refer for local UI actions and the redrawing
activities to the local data copy emplaced therein. It is recommended to define the actual DTO data
type in such a way that a default constructed DTO reflects the _neutral state_ of the corresponding
UI facility. This way, the UI will always come up with a valid representation, even while the core
has not (yet) sent actual data updates.
Implementation of data transport
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Actually, GuiDataExchange acts as a mediator, it maintains a registry and uses the existing
command-system of InterChange to send _notification events._ For the data to be published, internally
there is a circular set of _temporary data buffers._ As of 3/2024, the number of data buffers is
a compile time constant (see GuiDataExchange.cpp, the local constant `CAP`). Moreover, at the moment,
only a single »slot size« is maintained, established at compile time to be large enough to hold the
largest data type to be pushed through this communication system. If memory consumption turns out
to be a problem, these implementation details could be changed without impact on the usage of the
system.
Whenever a new update is »pushed« into some connection channel, GuiDataExchange will claim the next
free buffer slot, and then just send the index number of this slot as a notification message through the
command system, using `control = TOPLEVEL::control::dataExchange` and `part = TOPLEVEL::section::message`
and placing the slot index number into the `offset` field of the `CommandBlock` structure.
It is assumed that these notification messages will be received within another thread (the UI thread).
Actually this is done in `GuiUpdates::decode_updates()` (see MiscGui.cpp). Next, a dispatch must be
requested, by invoking `GuiDataExchange::dispatchUpdates(CommandBlock*)` within this UI event thread.
The implementation will then do a lookup of _all subscribers currently enrolled with this channel;_
it will pick up the data from the designated buffer slot and directly place a copy of this data
into the MirrorData block of each subscriber. Note however, since this dispatch on notification
happens asynchronously and typically slightly delayed, there is no guarantee that the buffer slot
is still valid; if too many push updates are sent before notifications are received and dispatched
on the UI side, then data may be lost. In such a case, the number of buffer slots could be increased.
There is a safeguard in place however, as the data type-id of the data found in the designated slot
must match up with the data type expected by the Subscriber.
Bootstrap of the UI
~~~~~~~~~~~~~~~~~~~
To bring up this communication scheme, initially the partners on both ends must know about the (opaque)
connection ID. In theory, hard coded IDs could be defined and maintained somewhere in the code.
However, doing so would be tedious, require constant attention and is thus unnecessarily error prone.
It seems more adequate to just start with a connection object in the core (which gets a distinct ID)
and establish the connections by passing the routing tags. This way, we have only to connect variable
definitions together. These can be named to be self-documenting, and moreover the compiler can check
that the data types delivered through any connection are consistent.
To bring up such a semi-automated connection system, a special DTO is introduced: the `InterfaceAnchor`
When the GUI starts up, a copy of the InterfaceAnchor is placed into a slot in GuiDataExchange,
and the MasterUI object is initialised with the slot number, to establish the first connection.
From this point on, updates to the InterfaceAnchor could be sent any time by regular push updates.
Since all UI windows and relevant controls are more or less directly established going down from
MasterUI, the suitable fitting connection objects can be passed directly to the init() function of
all UI components in need of direct data push updates. Thus MasterUI holds a MirrorData<InterfaceAnchor>
in a public member field `GuiUpdates::anchor`, and all further parts of the UI can assume to find
valid connection objects ready to be picked up through `uiMaster.anchor.get()`
|
