Data not saved due to Pavlovia Gitlab error

URL of experiment:
https://run.pavlovia.org/denispelli/perception2/
Description of the problem:
EasyEyes is built atop of PsychoJS. I ran an EasyEyes experiment in class with 20 students. I gave them the study URL (above), which they typed into their browsers. It worked fine for nearly everyone. One student, on a MacBook Pro got this fatal PsychoJS error (see screenshot), which is new to me. My hunch is that Pavlovia’s Gitlab couldn’t handle two instances of PsychoJS (i.e. two students) uploading data at once. Is that right? Ideally PsychoJS would pause and try again instead of just dying without saving the data.
P.S. When I downloaded all the data I did find a 606KB CSV data file for CrazyForestPear249, so it appears that most of her data were saved. Phew! Even so, the fatal error is worrying. If she’d been recruited online she wouldn’t be sent back to Prolific with a completion code, and might not get paid.

TEXT FROM SCREENSHOT:
Unfortunately we encountered the following error:
• when uploading participant’s results for experiment:
denispelli/perception2
• when saving data from a previously opened session on the
server
• unable to push data with key:
CrazyForestPear249 Threshold 2022-02-
02_16h02.27.967.cv to GitLab repository: Cmd(‘git)
failed due to: exit code(128) cmdline: git add
/var/www/pavlovia.org/run/denispelli/perception2/data//Cra
02-02 16h02.27.967.csv stderr: "fatal: Unable to create
"/data/run/denispelli/perception2/.git/index.lock’: File
exists. Another git process seems to be running in this
repository, e.g. an editor opened by ‘git commit’. Please
make sure all processes are terminated then try again.
If it still fails, a git process may have crashed in this
repository earlier: remove the file manually to continue.’
Try to run the experiment again. If the error persists, contact the
experiment designer.

I agree that this is concerning and has happened before.

I think that it may be better when you save to database instead of CSV because fewer files are being saved to gitlab.

Thanks!

Hi @wakecarter I was wondering if you could share any more insights on this ‘saving as database vs saving as csv files’ issue. We are about to commence collecting data in schools with lots of kids completing the tasks simultaneously, and want to do everything we can to avoid data loss due to conflicts at the data saving stage. Last night, based off your comment here, I did a bunch of testing comparing csv and database saving methods, and implementing small breaks in the internet connection. Indeed, saving as a database seemed more reliable, as when saving as csv files some files were missing after the internet connection break (rather randomly). However, this morning when doing the same kind of check, csv files are saved properly as well. So, I am a bit puzzled and hoped you might be able to share you knowledge of how the data saving all works so we can make the best choices.
Best wishes,
Donna

Hi Donna,

I don’t have any good insights into exactly how the data is saved. I suspect that there isn’t likely to be an issue when you have a class of participants (e.g. 30-60) starting at the same time.

I have had more issues with the shelf where the record sometimes gets “locked”, preventing future participants from using it.

1 Like

Dear @dbryce ,

Under some (hopefully rare) circumstances, the .csv files might be missing from the GitLab repository, but I believe they would have been saved to your experiment repository on the Pavlovia server (as opposed to the Pavlovia GitLab server). Consequently, they would be available to you through the [Download results] button on the Pavlovia page for your experiment. Could I ask you to check whether that is indeed the case?
If you could you share the full path of the experiment(s) you ran for your tests, I could also check on my end.
Best wishes,

Alain

Hi Alain, thanks for your response. Unfortunately I cannot check that anymore, as I have since cleared all the data. But it is good to know that this is the first thing we should check if datasets are missing in gitlab. Do I understand correctly that based on your expertise there is no reason to prefer either csv or database as our mode of data saving? Another idea I had was that we take control of ending the experiment for each child (i.e. the research assistants press the button to end the experiment one after another, perhaps even after checking internet connection is good). Would this reduce the chances of such a conflict, because there is no way two experiments could be ended at exactly the same time? (In case it is important, we are testing on Microsoft Surface Pro tablets.)
One other question, is there any difference in the resulting logfiles depending on whether Logging Level is Exp or Info?
If you have any other tips for minimising our chances of data loss I would be very grateful!
Best wishes,
Donna