Timing mega-study: comparing a range of experiment generators, both lab-based and online
Audio, visual and response timing, on many packages, OSs and browsers.
On the desktop: PsychoPy, Psychtoolbox, Presentation, E-Prime, Open Sesame, and Expyriment
Online: PsychoPy, GorillaPsyc, jsPsych, Lab.js, and Testable
- Best timing is on desktop still, but online studies getting better.
- PsychoPy in Python achieved sub-millisecond precision almost across the board (except Mac visual presentations). Similar timing for Psychtoolbox, E-Prime, Presentation. Less good timing in OpenSesame and Expyriment.
- PsychoPy online (version 2020.1) achieved RT std dev under 3.5 ms on every browser/OS combo!! (Other packages had online RT std dev under 10ms in most cases)
- Although variability (std dev) is generally low online, lags are bigger (for all pkgs)
- None of the online experiment packages really manage good audio timing as yet (consistently across browsers)