Random online experiment crash/freezes, potential memory leak, Psychopy 2021.2.3

Online experiments
1

URL of experiment: Biomotion_rating_pilot [PsychoPy]

Description of the experiment:
This is a fairly simple rating study where participants are asked to rate 126 videos, length varies between 3 and 13 seconds. The video resource is downloaded on the fly and once the download is ready, participants can start the next trial. Once the video has played, the response screen appears. This will ask participants to rate each video on 4 dimensions using the slider component. The appearance of the 2nd/3rd/4th rating scale is code to be conditional on a response being given to the previous scale and participants do have the option to click a button to rewatch the video at this stage too.

The experiment takes about 45-60 minutes to complete.

Description of the problem: In my own tests on Mac 10.15.7 (Catalina) in a Chrome, Version 103.0.5060.53 (Official Build) (x86_64), I have not had any of these issues when developing the task. Other lab members also tested it without issues. However since data collection (both on SONA & Prolific) began, there are several participants out of a relatively small sample that have experienced one of the following

  • random task freezes with the screen just being the black background either where the next video does not play or at the end of the rating scale
  • relatedly one participant said that he got through after 2 hours because of serious slow down and lag and freezes, using a Windows NT 6.3 system with Chrome
  • others have reported that the task randomly restarted and the screen going white, showing the again the ‘Initialising’ screen
  • one participant reported a ‘low memory’ error message but didn’t provide a screenshot or further details, according to Qualtrics data this person also used Windows NT 6.3 so an older version with Chrome
  • a more unrelated error which nonetheless occurred quite a bit: people started the study and it supposedly just skipped to the end despite being in full-screen mode. I have a code component that would skip all experiment routines and go to the end if people exited the experiment early but those people who got back to me confirmed that they were still in full-screen mode

I have checked and the freezes aren’t associated with a particular video. All videos are .mp4 H.264 codec. The video file sizes ranges from a bit more than 100KB to at most 1.2MB.

In sum out of 19 people on Prolific alone, at least 7 of them were in touch with me about issues described above and have resulted in slower experiment times, loss of data etc despite having to pay people which is really frustrating. Most of them used Windows NT 10.0 (except for the two mentioned above with older Windows) and Chrome 103.0.0.0.

I have looked at lots of older threads taking about memory leak issues in an older 2020 version so I was hoping this would be fixed in the version I am using. Thus, I have no clear idea how to address this issue and would really appreciate help from the PsychoPy/Pavlovia technical experts.

Thanks!

2

Hello,

well, Windows NT 6.3 is really old (no support since 2018, extended support will end some time in 2023).

If you don’t find a problem running the experiment online on more recent win-PCs you might want to exclude participants on the basis of the os. If you can’t recreate the errors, you might want to go this way.

Best wishes Jens

3

Thanks yes I thought that too but to clarify, the other people who had issues had all Windows NT 10.0 including people who reported freezes so I don’t think that’s the problem. Especially given that they all used the same version of Chrome.

4

Hello Julia,

you can also check for the browser your participants are using :wink:

Best wishes Jens

5

Yes I have and it’s the same browser for people with and without crashes. Apologies, I didn’t mention that in my reply but did in my edits to my original post to reflect your point.

6

Hi There,

Sorry to hear that you’re having these troubles. Movie playback if something we are aware could be smoother (in particular because as you describe studies with movies can get quite resource heavy on the browser!) the team is working on improving this and I’ll share this post with them. For now, here are some tips I’ve found useful:

  • Ensuring all movies have a size in the experiment file (the black background issue you describe has happened to me when my movie didn’t have a size).
  • In general we’ve found firefox to be the most reliable for movie playback.
  • If you can reduce the size of movies I’de recommend doing so
  • if you can split your task into two smaller tasks that are daisy chained, this can help minimise data loss, as data will be pinged back to the server at the daisy chain.

Becca

1 Like
8

Hi Becca,

Thanks a lot for your reply and advice! It’s much appreciated.

  • I will definitely address the size issue. I just wanted them in the original size 800x600 as they were centrally presented but I’ll add the size to avoid having that as a undefined. I did notice the warnings about that in the console actually.
  • Also thanks for pointing me to Firefox. I didn’t know that :slight_smile:
  • I’ll see whether I can compress the files further and could check whether the crashes happened with longer files most of the time.
  • I should be able to split my task into two 30 minute sessions, although I would want to avoid assigning specific stimuli to be rated only in the first or second part so I’l have to figure out how to implement the randomisation manually.

Thanks again,
Julia