Timing and empirical testing of Expyriment
==========================================

**How accurate is the timing in Expyriment?**

In general, the Expyriment clock can feature up to 1 ms accuracy. The exact 
timing, however, is subject to various factors, which we discuss here.

Time and compatibility issues can be conveniently tested using the 
:doc:`Expyriment test suite <Testsuite>`.


Stimulus presentation
---------------------

Visual
~~~~~~
Computer screens are updated according to their refresh rate. What happens then 
is that the whole screen is redrawn line by line all the time. For example, 
with a refresh rate of 60Hz, the screen is redrawn 60 times per second (1000ms) 
and the duration it takes to redraw it line by line is 16.66 ms (1000/60).
When attempting to redraw the screen while it is currently already being 
updated (the lines are drawn) the result might lead to artifacts, since the 
update occurs immediately, leading to parts of both, the new and the old screen 
content, being visible. What is even worse is that you will never know in which 
phase of the redrawing the new redraw was started. Thus, you cannot be sure 
when exactly the new content is fully visible on screen.

The first step towards getting around this problem is to synchronize the actual 
redraw to the vertical retrace of the screen. This means that a change in 
content will never happen immediately, but always only when the retrace is at 
the top left position. When synchronizing to the vertical retrace, the graphic 
card is told to update the screen the next time it starts redrawing the first 
line. While this will solve the problem of artifacts, you will still face the 
problem of not knowing when exactly something was visible on the screen, since 
the graphic card handles this synchronization itself in the background.

Solving this problem is the exact (and only) reason why Expyriment uses OpenGL 
by default. It allows to wait for the vertical retrace to actually happen 
before proceeding with the code that tells the graphics card to update the 
screen (this is also known as blocking on the vertical retrace). This means 
that whenever something should be presented on screen, no matter in which line 
the redraw is at this point in time, the graphic card will wait for the redraw 
to be in the first line and then present the stimulus. Since the code is 
blocking, the time Expyriment reports the stimulus to be presented on screen 
will always be the time when the redraw is starting at the first line. Coming 
back to the example of the small dot in the center of the screen: Expyriment 
will correctly report a longer presentation time when the redraw has been just 
over the center line when the screen update was issued.

*It is important to set your graphic card's driver settings to support 
synchronizing to the vertical retrace ("Sync to VBlank" or "V-sync") and to 
switch off any power saving schemes on the graphic card.*

**Test results**

We tested the visual timing by presented a white and a black fullscreen 
rectangle directly after each other. The brightness of the left upper edge of 
the screen was recorded using an optic sensor attached to an oscilloscope.  
After each screen presentation, a marker was send via the serial port to the 
oscilloscope. Testing was done on an Intel Core Duo PC with an Nvidia Quadro 
NVS 290 graphics card, running Microsoft Windows XP SP3. The monitor used was a 
Samsung SyncMaster 2233. Expyriment was running in OpenGL mode (default).

The results revealed:

* When presenting a stimulus, updating the screen does successfully block code 
  execution until the vertical retrace actually happens. This can be seen in 
  Figure 1, where the marker (yellow) lines up with the onset and offset of an 
  increase in brightness (turquoise) which represent the onset of the white and 
  the onset of the black screen, respectively.
* Presenting (preloaded) stimuli can accurately be done each refresh rate 
  (Figure 2).

Visual stimulus presentation is time locked to the vertical retrace.

.. image:: timing_visual1.png
   
Visual stimuli can be presented each refresh rate.

.. image:: timing_visual2.png


Audio
-----
Playing back audio is handled by PyGame. The present() and play() methods of 
auditory stimuli will return immediately. Since the audio stream has to be sent 
to the hardware, there will still be a delay before the audio can be heard.  
Unfortunately, the latency of the sound onset is not known by Expyriment.  
However, it is assumed to be relatively stable over time. Setting the audio 
buffersize to a smaller value than the default can decrease the delay, but 
might result in distorted audio.

*It is important to set your samplerate, bitdepth and audio buffersize 
correctly. Setting the buffersize too low will result in distorted audio!*

**Test results**

We tested the audio timing by repeatedly playing back a beep tone (a 1 second 
sine wave). The output of the sound card was measured by an oscilloscope.  
Before starting playback of the beep, a marker was send via the serial port to 
the oscilloscope.  Testing was done on an Intel Core Duo PC with a Soundblaster 
Audigy sound card, running Microsoft Windows XP SP3. In Expyriment, the 
samplerate was set to 44100 Hz, bitdepth to 16 bit and the buffersize equaled 128.

The results revealed:

* Audio playback was subject to a latency of maximally 20 ms. Figure 1 shows 
  the maximal measured latency between the start of the playback (yellow) and 
  the onset of the sound (turquoise).
* This latency was relatively stable with a jitter of 5 ms. Figure 2 shows the 
  minimal latency we could measure.

Maximal measured audio latency.

.. image:: timing_audio2.png

Minimal measured audio latency

.. image:: timing_audio1.png

Video
~~~~~

Video presentation is a tricky subject. In Expyriment, the present() method of 
a video stimulus will start playback and present the first (current) frame on 
the screen. Thus, visual onset of this frame will be synchronized with the 
vertical retrace (see visual stimulus presentation above). Each following frame 
has to be plotted on the screen and the screen has to be updated. The 
wait_end() method of a video stimulus will automatically present each frame on 
the screen until the video is over. When Expyriment is in OpenGL mode, the 
process of plotting each frame might take longer than one refresh rate which 
will result in dropping frames (e.g. frames not being presented at all). To 
control for this, the wait_end() method will report and log if any frames were 
dropped during video playback.

*Unfortunetly, right now, Expyriment can only handle MPEG-1 encoded videos!*

Measuring user input
--------------------

In Expyriment all inputs (keybard, mouse, gameport, serial port, parallel port) 
can be checked by directly polling them (via the wait() methods of the 
corresponding io object). This allows for the most accurate timing possible.  
Since Python wraps C functions for getting the system time, the accuracy is 
even more precise than milliseconds (which is the unit Expyriment uses).

Expyriment does *not* have a main event loop (i.e. it will not automatically 
check for any incoming events in the background)! This was a design decision, 
since we think that in 99% of all cases the time of the user input is specified 
in the design and thus know beforehand (e.g. a response after a stimulus 
onset). Adding an event loop would make things unnecessarily more complicated 
for those 99%.

However, we also thought of those cases that need to check user input during 
other operations: All events can manually be pushed from either Pygame's event 
cue (keyboard, mouse, joystick) or the operating system's buffer (serial port, 
parallel port) into an EventBuffer object. Doing this regularly is up to the 
user.

Keyboard
~~~~~~~~
Keyboards (PS2 and USB) are known to have poor timing accuracy. Usually these 
are in the range of several 100th of a second.

**Test results**

We tested the timing of a Logitec USB keyboard in Windows XP SP3 using optical 
tracking.  

Our results revealed:

* A timing accuracy between 20 and 26 ms.

Mouse
~~~~~
On most operating systems, USB mice are polled at a rate of 8 ms. Mice with 
special drivers might be set to poll more often.

**Test results**

We tested the mouse accuracy of a standard USB mouse on Windows XP SP3 by 
measuring the time between reported position changes.

Our results revealed:

* The expected standard accuracy of 8 ms.
* Using a Logitec G700 USB mouse with a dedicated driver, polling rates could A  
  be reduced, leading to an increased accuracy of 1 ms.

Serial port
~~~~~~~~~~~
The serial port is very accurate and thus suited for timing accurate 
measurements. If a computer does not have a serial port, USB-to-serial 
converter can be used (e.g. from Sweex or Keyspan). However, the timing 
accuracy of these depends on the implementation and drivers used!

*It is important to deactivate any additional FIFO buffers or delays, provided 
by the port driver!*

**Test results**

We tested the timing of a UART 16550A serial port (a real one, not a 
USB-to-serial converter!) on Windows XP SP3 by sending a byte to a connected 
loopback device which immediately sends the byte back. We then measured the 
time between sending and receiving. We repeated this process 1000 times.

Our results revealed:

* With a baudrate of 115200, the maximal measured time between sending and 
  receiving a byte was 0.283894736842 ms.
* With a baudrate of 19200, the maximal measured time between sending and 
  receiving a byte was 0.689593984962 ms.

Parallel port
~~~~~~~~~~~~~
The parallel port works by directly applying a current (writing) and measuring 
if a current is applied (sending) to several pins on the connector. Expyriment 
is only able to read from Acknowledge, Paper-Out and Selected pins!