I am running an online study in Prolific (developed the program in PsychoPy 2022.2.4 and adapted to Pavlovia using jsPsych. We are recruiting through Prolific – participants are directed to the experiment link on Pavlovia and at the end of the study they get a completion code on the final page that they enter into Prolific to confirm completion. We are saving the results as CSV files on Pavlovia.
We did a pilot of just 6 participants to make sure all was running smoothy - all six participants finished the study and entered the completion code in Prolific. However, when we download the data from Pavlovia, there are only four complete data files (csv with responses + log file). The other two participants’ data saved as empty csv files and no log files were created. I messaged them on Prolific and already heard back from one, who was able to describe the entire study (and had successfully entered the code from the final page), so I know that they did the whole experiment. They said they did not notice anything strange or receive any error messages.
We have wondered if it was an issue of them exiting too soon (i.e., not seeing the “thank you for your patience” message at the end) after reading other posts. However, we have the “Save incomplete results” option toggled on. We just did our own test where we exited the study after a few trials only and Pavlovia successfully generated a csv file saving the partial data as expected.
Does anyone have ideas of what is going on and what we can change to avoid this in the future? Based on this pilot, we are losing 33% of our data and that is obviously not sustainable for an experiment where we will be required to pay hundreds of participants whose data we do not have.
The empty csv files occur when the browser tab (not the whole browser window) has been closed, and then “Leave” has been clicked on the pop-up alert that appears. This alert says “Leave site? Changes you’ve made may not be saved”, and if Leave is clicked then the data isn’t stored. I imagine what’s happened is that the participants have left the Pavlovia tab open whilst they copy the completion code into Prolific but then closed the Pavlovia tab when they’ve done that. It might be worth popping a message into your study that alerts them to this and tells them to make sure they have seen the “Thanks for your patience” pop up appear before closing their tab or window.
Note that them actively choosing to click leave despite being warned about their data not being saved is different from what happens if the ESC key is pressed - if the ESC key is pressed part way through an experiment, the incomplete data is saved.
Hi @Kimberley_Dundas , thank you for the elaborate response. I’m facing a similar issue despite witnessing the participants pressing enter to upload their data to the pavlovia server. Is there any alternative cause to this issue? Any guidance would be greatly appreciated.
Did you witness the green barred “Thank you for your patience” message which comes up when data has been saved?
Do you have any End Experiment code (which might interfere).
I haven’t yet seen proof of data loss at this stage without an error message. The VESPR Study Portal can be used to confirm that participants completed.
@wakecarter thanks for your swift response. As participants are using laptops and PC’s, the message was that the data has been uploaded successfully. Attached are sequential screenshots of what happens at the end.
The first screen is a part of my experiment (completionCodeTrial on the timeline)
The second screen appears when the participant presses (pavlovia_finish on the timeline)
The third screen shows up after the second screen without any prompting from the user
When I download the csv file associated with this participant, it looks like this
This is the code that I’m using for the second screen:
var pavlovia_finish = {
type: jsPsychPavlovia,
command: “finish”,
participantId: “JSPSYCH-DEMO”,
// Thomas Pronk; your filter function here
dataFilter: function(data) {
// Printing the data received from jsPsych.data.get().csv(); a CSV data structure
console.log(data);
// You can also access the data directly, for instance getting it as JSON
console.log(jsPsych.data.get().json());
// Return whatever data you’d like to store
return data;
},
// Thomas Pronk; call this function when we’re done with the experiment and data reception has been confirmed by Pavlovia
completedCallback: function() {
alert(‘data successfully submitted!’);
}
};
Are you using PsychoPy/PsychoJS or jsPsych? If it’s the latter then I’ll move your thread into that category and recommend that you search it for ideas.
