Partial data loss in Pavlovia - empty csv, no log file

Description of the problem:

I am running an online study in Prolific (developed the program in PsychoPy 2022.2.4 and adapted to Pavlovia using PsychoJS. We are recruiting through Prolific – participants are directed to the experiment link on Pavlovia and at the end of the study they get a completion code on the final page that they enter into Prolific to confirm completion. We are saving the results as CSV files on Pavlovia.

We did a pilot of just 6 participants to make sure all was running smoothy - all six participants finished the study and entered the completion code in Prolific. However, when we download the data from Pavlovia, there are only four complete data files (csv with responses + log file). The other two participants’ data saved as empty csv files and no log files were created. I messaged them on Prolific and already heard back from one, who was able to describe the entire study (and had successfully entered the code from the final page), so I know that they did the whole experiment. They said they did not notice anything strange or receive any error messages.

We have wondered if it was an issue of them exiting too soon (i.e., not seeing the “thank you for your patience” message at the end) after reading other posts. However, we have the “Save incomplete results” option toggled on. We just did our own test where we exited the study after a few trials only and Pavlovia successfully generated a csv file saving the partial data as expected.

Does anyone have ideas of what is going on and what we can change to avoid this in the future? Based on this pilot, we are losing 33% of our data and that is obviously not sustainable for an experiment where we will be required to pay hundreds of participants whose data we do not have.

Thank you in advance for any ideas!!

Hi @famalab,

The empty csv files occur when the browser tab (not the whole browser window) has been closed, and then “Leave” has been clicked on the pop-up alert that appears. This alert says “Leave site? Changes you’ve made may not be saved”, and if Leave is clicked then the data isn’t stored. I imagine what’s happened is that the participants have left the Pavlovia tab open whilst they copy the completion code into Prolific but then closed the Pavlovia tab when they’ve done that. It might be worth popping a message into your study that alerts them to this and tells them to make sure they have seen the “Thanks for your patience” pop up appear before closing their tab or window.

Note that them actively choosing to click leave despite being warned about their data not being saved is different from what happens if the ESC key is pressed - if the ESC key is pressed part way through an experiment, the incomplete data is saved.

I hope this helps!


1 Like