Description of the problem:
I have a multi-task experiment that participants run through (programmed in jsPsych). For one of the tasks (snsl, the first in the chain of tasks, and not the other) the data fails to save approximately 20% of the time (7 out of the last 25 participants run today). The participants report no errors - and they continue on to the next task (mtrxiq) and that data always saves. There is no consistent browser or platform identity that distinguishes them from other participants (and again - their data save just fine for the second task).
This has cost me close to $100 of participants at this point, and I’m pretty concerned.
I am happy to try that - though I would note that both programs use the 3.0.6 version of the plugin and only one is having issues with saving - so it would have to be some sort of interaction, not simply an update issue.
Thinking about it, I am wondering one difference between the 2 programs is that there is quite a bit more data in one (1600-1700 trials) than the other (20-30 trials). I would be surprised if this was an issue, but perhaps there could be some interaction between a user’s connection speed and the fact that the first program redirects them to a second program (at a different url) automatically at the end? is it possible they could be moving ahead too quickly in some cases (i.e. before the data downloads)? That seems pretty far-fetched given the amounts of data we’re actually talking about (~500kb), but I’m not sure what the plugin is doing on the back end.
I will try a small batch today (generally preferred in my bourbons, not my data collection )
P.S. - I’m assuming that changes between the jspsych plugin 3.0.6 and 3.1.0 are under the hood and backwards compatible so that it’s a simple matter of updating the link to the library?
I would also note that I have chained tasks together in this manner before (using earlier versions of the jsPsych plugin) and I have never had a problem - so perhaps my musings above should be disregarded…
Hello - This is still an issue with the new jsPsych plugin. I am having a research assistant test it - but it may be related to internet connection strength - does that sound plausible? Is it possible for the experiment to advance past the pavlovia plugin if data-saving is not successful? If so - is there a way to build in a data-uploaded flag?
So - it now seems relatively clear that this is a replicable issue. I have a student testing it and it has failed multiple times to upload the data for the SNSL task (but not the MTRXIQ task). This is not a data issue - as I have now set the programs to also download the data to the local computer.
My very hacky and unsatisfying solution at this point is to figure out a way for my participants to anonymously upload files to me if/when their data do not save. If anyone knows of a good/simple way of doing this - please do share.
My suspicion is that this is the result of a temporarily failed or weak connection - and/or some interaction between having to upload a larger data file while also moving on to the next task. If you have thoughts about how I might alter my code so that this won’t happen - I’m more than happy to try.
I am sorry to read that you lost some participant results. Thank you very much for such a thorough overview of the issue, it has proved extremely useful.
I believe that the issue has to do with the fact that you are chaining experiments indeed.
The jsPsych pavlovia plugin communicates with the pavlovia.org server in an asynchronous manner. So, when your jsPsych experiment finishes, the _finish method of the plugin is called and starts doing its work. But the jsPsych script does not wait for it to be done. This is perfectly find under ordinary circumstances. But since the next step of your jsPsych script is to move onto another jsPsych script, it is entirely possible that the move happens before _finish is done and, in particular, before _finish has sent the experiment results.
Could I ask you to do the following test: wait for 60s before pressing space at the end of the first experiment in order not to immediately move onto the next experiment. I strongly suspect that you won’t have any issue with missing data under those circumstances. In the mean time, I’ll work on a synchronous call to prevent the issue to appear in the first place.
Cheers,
Thank you for your response! I will try inserting a pause before participants are allowed to continue. I realize that 60 secs is a conservative estimate - though it’s a long time to ask participants to wait. The average data file size is ~500Kb, given what you know about the process, what is the shortest time you think I could get away with and still be safe?
60s is indeed probably much too long for participants. I was suggesting we use a 60s pause as a test, to confirm my diagnosis. Meanwhile I’ll develop a synchronous approach, which is the correct way to solve the problem.
Indeed - I will be having my RA wait 60 seconds (I have inserted a pause screen at the end of the task) during testing. I very much appreciate that you will develop the appropriate fix. I’m not sure how long that will take - and I need to collect data in the next few days. So in the meanwhile I plan to implement a kludgy fix to enable data collection.
If you have a guesstimate estimate of how long you think should be enough, then I’m happy to move forward with a real data collection (I will be also instituting a local save until we know this is fixed - so no data will be lost). That way I can continue to collect data. If you don’t feel comfortable implementing a guess, no worries, I will have participants wait 60 seconds (in an hour long experiment it’s not that big a deal)
Note: 60 second pause (implemented with an inserted trial) has been effective for a couple of tests - however it only happens sporadically, so we cannot know for certain.
I am glad to read that the issue has not occurred again, at least not yet, with the 60s pause. I do very much believe that the loss of data you experienced had to do with the asynchronous data saving calls being interrupted, as mentioned earlier.
I have now reviewed the plugin situation. Alas, I don’t think there is much I can do: jsPsych does not have a callback mechanism I could use to tell it to wait for data to be saved (incidentally, I believe that also applies to their own approach to saving data), and synchronous communications are either strongly discouraged or deprecated in modern browsers (the Beacon API would sort of help but is not powerful enough for our purpose).
What I would suggest you do is either:
(a) introduce a pause, as mentioned already, or
(b) save data from your first experiment to the local or session storage on the browser, and read them from there, in the second experiment. You can read all about it here: https://developer.mozilla.org/en-US/docs/Web/API/Window/sessionStorage
The first option is of course slightly inconvenient for the participants and does not guarantee that the data will be saved. Consequently, I would recommend you use the second one. How does that sound?
I suspect I may be having the same issue, whereby the call to save the data is either not being executed or is happening in a manner that prohibits the successful saving of my data onto Pavlovia’s servers. I have programmed my experiment in jsPsych and it runs well and even saves my data locally when using the jsPsych.data.get().localSave command in my browser. However, Pavlovia refuses to save my data and aborts each and every run I do of my experiment despite my having completed it. When I go to close the tab, it also stutters, as if it was in the middle of doing something or some function was occurring that is invisible to me. It’s very strange and I don’t know what it could be. What’s worse is that I cannot troubleshoot the issue because I don’t get any error messages in the console. I’m wondering if perhaps my use of nested timelines within each trial is an issue… or is it just the fact that my script is over 600 lines? I really don’t know how to troubleshoot this issue and suspect it may be something on the back end that’s causing my issue. Here is my experiment, I would greatly appreciate it if you could take a look at it and let me know if this is hopeless or if some solution exists! https://run.pavlovia.org/andreja/inverted_upright_nomask/inverted_upright_nomask.html
I also experienced issues with this, which also cost me data on more than 5 participants. It’s really tough to deal with this when you are already struggling to find participants.
I also noticed that in my data folder on pavlovia there’s incomplete csv files which may indicate that the participants aborted the session but their data is still available. I thought this was not possible.
Yes, indeed, we’ve noticed that this is happening too. There have been some crossed wires about what should happen here which we’re currently sorting out. Very soon you will have an option on your Dashboard to select (on a per-project basis) whether you want to receive partial data on abort (and consume a credit) or not receive data (and not consume credit).
)