Discussion: What to do with partial data when a participant quits?

There have been a number of posts with users being frustrated by data being lost when participants exit the study before the end.

Of course, it’s understandable that users want to keep as much data as possible, including data from a part-finished task.

On the other hand, my personal feeling is that it’s essential that participants have a way to exit and have their data deleted - the vast majority of ethics boards/applications would have that as a requirement. Whereas in the lab participants can verbally state they wish to quit, when online they can’t communicate with the experimenter and the obvious way to exit (and delete data) is simply to close the tab.

There’s another issue about whether you, the user, wants to have the data from a partial run, which would then consume a credit. You might decide that you do want it (especially if you have a site license) but you might feel you only want to use your credits on full data sets.

The question is how to balance these desires, to maximize data collection in an ethical environment.

Here’s my proposal for how we continue, providing you more options (on a per-project basis on your Pavlovia Dashboard?):

  • I suggest we provide these options for User exits with Esc key:
    • Delete partial data (don’t consume credit)
    • Keep partial data (consume credit)
  • What to do when a Participant closes the tab:
    • Aways quit (delete partial data and don’t consume credit)
    • Attempt to warn user that data are about to be lost and offer them the options including save partial data (potentially consumes credit for partial)
    • Attempt to warn user that data are about to be lost and offer them the options but not including save partial data

The closing-tab options presented to participants may not work unifromly - some browsers may not allow it, either now or in the future, because of the possibility of creating malicious websites that prevent users from closing tabs. But it would be reasonable to try.

Do people have other thoughts? Are there better solutions? Have you seen something on other platforms that’s better? Should we be allowing you full flexibility to store data without the user being able to exit privately at all (on the basis that it’s your ethics approval being used)?

let’s discuss

On Qualtrics students are often surprised that partial data is available. They have to manually delete it based on a column in the data file called “Finished”. So, if Pavlovia was to allow access to partial data, there should be a similar column – ideally on every row of the data file. The most useful information would be for it to say Finished if the participant did finish or the name of the final routine if they didn’t.

Since there are many people out there routinely asking their participants to press escape on the final routine, I don’t think escape can be taken as a definite withdrawal of consent. Could there be a routine that runs when Escape is pressed which asks the participant whether they would like their data to be deleted.

One option is to make partial data only available to site licence users. It would be an added benefit for users with site licences and also avoid PAYG users from having their credits wasted.

Best wishes,

Wakefield

1 Like

An additional “Finished” column sounds like an excellent idea.

Could you elaborate on your comment about participants being asked to press Escape on the last routine? Do you mean that those experiments never actually finish but are interrupted at the last moment?

Here’s the thread where I discovered that pressing the escape key was sometimes being recommended as the correct way to end an experiment:

It makes perfect sense locally. You have a thank you message in your final routine with no method to continue so you can say goodbye to the participant without them seeing the PsychoPy builder window. Then you press escape and restart for the next participant. I think it’s a bad habit that has been learned by users of PsychoPy since data is saved as you go, so there is no difference between pressing escape or ending the final routine on a key press or time out. The bad habits of existing PsychoPy users need to be taken into account in this issue.

I also think that this post should be added to the current discussion