On 2016 Aug 28, Steven Finney commented:

Schultz and van Vugt (2015) (henceforth SVV) compare their Tap Arduino device against a PC-based system using the FTAP software package (Finney SA, 2001a), and claim that FTAP has high latency and variability (M = 14.6 ms, SD = 2.8). This claim is incorrect: as SVV show later in their paper, the single specific percussion pad that they used in their FTAP condition is responsible for the bulk of their measured latency, not FTAP. SVV also contains a number of additional false and misleading claims.

FTAP is a software package that runs on a Linux PC; it must be attached to a MIDI input device (a keyboard or percussion pad) for input, and to a MIDI output device (a tone generator) for output. The end-to-end latency of an experimental system using FTAP (as measured by SVV) will be the (linear) sum of the input device latency, the output device latency, and the latency of the FTAP-MIDI system itself. As reported in Finney SA, 2001a and Finney SA, 2001b, an FTAP-MIDI system running on a standard 200 MHz Linux PC processed MIDI data with millisecond accuracy and precision, and this is easily replicated on current hardware (SVV themselves report such a result on p 5 of their article, and further data is provided on the FTAP web page at http://www.sfinney.com/ftap). Since FTAP adds no latency that is relevant at the millisecond level, the "FTAP" latency reported by SVV must be due to their input and/or output device.

SVV thus err in confounding the MIDI I/O devices with the FTAP software. To compound the error, in their FTAP condition SVV used a single input device (a Roland HPD-15 percussion pad) that they knew to be defective (they report that it both missed responses and generated superfluous responses). They measure the audio latency of the percussion pad to be 9 milliseconds, and then repeatedly present the "FTAP" and "percussion pad" measurements as if they were independent, when the percussion pad latency is actually the major component of their "FTAP" latency. (This lack of independence in their conditions arguably invalidates their statistical measures). Their latency data thus demonstrate that their specific percussion pad was faulty but say nothing about FTAP.

SVV also repeatedly and mistakenly assert that USB and MIDI processing will add significant latency in a PC system (e.g., p 2, "delays can be introduced by the [USB] polling speed"; p 7, "temporal noise...probably due to the MIDI-USB and USB-MIDI conversions"). This claim is also disproved by their own data. The FTAP distribution provides a loop test that rigorously tests MIDI and USB input and output (both software and hardware), along with FTAP itself. SVV report (p 5) that their FTAP configuration had a loop test result of 1.01 ms; this shows that neither USB nor MIDI processing added latency that was significant at the millisecond level. (See Finney SA, 2001a, Finney SA, 2001b, and the FTAP web page for further discussion of the configuration and interpretation of the loop test)

Finally, SVV suggest that Tap Arduino is an inexpensive replacement for FTAP; this is also incorrect. Tap Arduino is simply a button box that can produce auditory feedback; it cannot run any experiments without being attached to a PC. SVV provide no data that such a complete Tap Arduino system has millisecond accuracy. In fact, they explicitly note that using Tap Arduino in a synchronization experiment has issues with the "asynchrony between the Arduino response and the onset of computer-generated audio" (p 11) that may require "expensive options" to solve.

SVV are to be commended for highlighting the importance of end-to-end measurement of systems such as FTAP and MAX, and for making a valiant, if flawed, first attempt at such measurements. However, the only useful thing that their article demonstrates is that the Roland HPD-15 percussion pad (with one particular set of configuration parameters) is not a suitable input device for millisecond data collection.

A validated Arduino-based MIDI input device that could be used with FTAP or MAX would be a valuable contribution; unfortunately, Tap Arduino is not that device.

On 2016 Aug 28, Steven Finney commented:

Schultz and van Vugt (2015) (henceforth SVV) compare their Tap Arduino device against a PC-based system using the FTAP software package (Finney SA, 2001a), and claim that FTAP has high latency and variability (M = 14.6 ms, SD = 2.8). This claim is incorrect: as SVV show later in their paper, the single specific percussion pad that they used in their FTAP condition is responsible for the bulk of their measured latency, not FTAP. SVV also contains a number of additional false and misleading claims.

FTAP is a software package that runs on a Linux PC; it must be attached to a MIDI input device (a keyboard or percussion pad) for input, and to a MIDI output device (a tone generator) for output. The end-to-end latency of an experimental system using FTAP (as measured by SVV) will be the (linear) sum of the input device latency, the output device latency, and the latency of the FTAP-MIDI system itself. As reported in Finney SA, 2001a and Finney SA, 2001b, an FTAP-MIDI system running on a standard 200 MHz Linux PC processed MIDI data with millisecond accuracy and precision, and this is easily replicated on current hardware (SVV themselves report such a result on p 5 of their article, and further data is provided on the FTAP web page at http://www.sfinney.com/ftap). Since FTAP adds no latency that is relevant at the millisecond level, the "FTAP" latency reported by SVV must be due to their input and/or output device.

SVV thus err in confounding the MIDI I/O devices with the FTAP software. To compound the error, in their FTAP condition SVV used a single input device (a Roland HPD-15 percussion pad) that they knew to be defective (they report that it both missed responses and generated superfluous responses). They measure the audio latency of the percussion pad to be 9 milliseconds, and then repeatedly present the "FTAP" and "percussion pad" measurements as if they were independent, when the percussion pad latency is actually the major component of their "FTAP" latency. (This lack of independence in their conditions arguably invalidates their statistical measures). Their latency data thus demonstrate that their specific percussion pad was faulty but say nothing about FTAP.

SVV also repeatedly and mistakenly assert that USB and MIDI processing will add significant latency in a PC system (e.g., p 2, "delays can be introduced by the [USB] polling speed"; p 7, "temporal noise...probably due to the MIDI-USB and USB-MIDI conversions"). This claim is also disproved by their own data. The FTAP distribution provides a loop test that rigorously tests MIDI and USB input and output (both software and hardware), along with FTAP itself. SVV report (p 5) that their FTAP configuration had a loop test result of 1.01 ms; this shows that neither USB nor MIDI processing added latency that was significant at the millisecond level. (See Finney SA, 2001a, Finney SA, 2001b, and the FTAP web page for further discussion of the configuration and interpretation of the loop test)

Finally, SVV suggest that Tap Arduino is an inexpensive replacement for FTAP; this is also incorrect. Tap Arduino is simply a button box that can produce auditory feedback; it cannot run any experiments without being attached to a PC. SVV provide no data that such a complete Tap Arduino system has millisecond accuracy. In fact, they explicitly note that using Tap Arduino in a synchronization experiment has issues with the "asynchrony between the Arduino response and the onset of computer-generated audio" (p 11) that may require "expensive options" to solve.

SVV are to be commended for highlighting the importance of end-to-end measurement of systems such as FTAP and MAX, and for making a valiant, if flawed, first attempt at such measurements. However, the only useful thing that their article demonstrates is that the Roland HPD-15 percussion pad (with one particular set of configuration parameters) is not a suitable input device for millisecond data collection.

A validated Arduino-based MIDI input device that could be used with FTAP or MAX would be a valuable contribution; unfortunately, Tap Arduino is not that device.