Hi
I think the problem might have been fixed now (I’ll copy and paste the email below, I’m sure there’s a discourse thread for it somewhere but can’t seem to find it!).
I haven’t tested since the update… but I think my issue was also probably due to participants exiting the programme prematurely e.g. if they press ESC or close browser before the programme ends, even if they’ve ‘completed’ the experiment, the programme would see it as the participant withdrawing. I did also get partial data though, so I’m not really sure whether there were other issues…
See email from PsychoPy below (I received it 27 June)
Hi there,
We are writing to inform you about an upcoming change to how we handle experiment results when participants fail to complete a study (‘incomplete runs’).
TL:DR We have introduced a new setting on the Pavlovia projectpages that determines whether data from incomplete runs are stored (and whether a credit is consumed in this case), taking effect on Tuesday, June the 30th.
Up until now, when a participant pressed the Esc key to interrupt a study, the partial data were saved (without a credit being consumed) whereas when a participant closed the browser tab we treated that action as consent withdrawal and, consequently, did not save the data (and did not consume a credit).
We have identified three potential downsides to this policy:
Did the participant intended to leave the experiment or merely closed the window by mistake?
Is it appropriate to keep the data from an incomplete run? We opened a discussion on the forum a month ago (Discussion: What to do with partial data when a participant quits? - #4 by wakecarter) and have come to the conclusion that this would best be left up to the researcher. We urge you to think carefully about whether a participant expects their data to be deleted when they withdraw, and what your ethics approval/information/consent form says about this scenario. If you intend to keep the partial data when your participants exit we believe that, morally, it would be best to inform them that you do. Legally, it will depend on your local rules and laws.
Should credits be consumed for partial data? Those among you working with Participant Credits rather than a Site License may or may not want to spend a credit for incomplete data.
We will be implemented the following solution:
- We will ask the browser to warn participants before they leave the page that data are about to be lost. You might have seen such warnings before when leaving a web page with a partially filled form. That technology is somewhat limited: it’s browser specific, we can’t change the text included in the warning, and if a browser doesn’t permit the message at all (now or in the future) we can’t do anything about that. As you may know, browsers are constantly evolving and changing what is or is not permitted, to make sure that programmers don’t abuse the functionality to prevent people from leaving their malicious site. At first, this will apply to the latest versions of the library (from 2020.1 onward). We will propagate this behaviour back to older versions in the coming weeks.
2 & 3. To make it possible for you to decide how to handle partial data, you will find a new control in the Dashboard, on the projectpage, in the ‘Saving Results’ section, asking you whether to save incomplete results. You can then decide to:
save incomplete results: a credit will be consumed for those of you not covered by a license
not save incomplete results: no credit will be consumed
That new control will take effect on Tuesday, June the 30th.
Best wishes,
The Pavlovia team.