Task crashed or froze for 30% of online participants

Online experiments
1

URL of project: https://pavlovia.org/AnthonyGabay/foraging_online:

Description of the problem:

Having finally got the task working online, I excitedly put it up on Prolific to start collecting data.

30% of my participants were unable to complete the task. There were separate problems being reported.

First, some people did not get past the “Initialising the experiment screen”. It just froze there. Second, people experienced a crash or random freeze. There seems to be no consistency in which part of the task it happened. Some made it all the way to the very end before it crashed, others crashed near the beginning, yet others somewhere in the middle. Some people described the crash as a frozen screen, some said the window went black.

I asked them all which browser and operating system they were using and all of them were using Windows 10 and either Chrome or Edge as browsers.

This is really frustrating because for those who made it part way through the task we obviously have to pay them, and we get no data at all when it crashes. Luckily I was only collecting a small sample of 25 people, but my next step is to scale it up to the hundreds. There is no way I feel comfortable doing that at the moment.

I not that a similar question was raised recently with no reply: Online study crashes on Windows Chrome

After all the effort that’s gone into getting this ready, I really don’t want to have to ditch psychopy and code my task in a different platform.

Does anyone have any idea what could be happening and how to address these issues when they don’t occur on the machine I’ve developed the task on?

2

Just an update from this. I have now heard back from another participant who said that he received an error message saying there was a problem saving the data. I’m going to assume it was the same as the problem I asked about in this question, which went unanswered: Error saving data when online

I thought I had fixed that because it didn’t reoccur after multiple tests on my own machine.

@dvbridges do you have any thoughts on this and my original post above?

Cheers,
Anthony

3

Hi Anthony, sorry to hear you’re having sporadic errors. I agree they’re annoying, and a little hard for us to debug until we can find a machine/browser where we can replicate the problem, but we’ll do our best.

Is it possible, on the machines where the participant just gets the message ‘Initiallising…’, that those were using Internet Explorer instead of Edge? That is a known issue, although the latest version of PsychoJS should at least be warning them that IE currently isn’t supported (we will hopefully be supporting it again soon)

4

Hi Jon,

Thanks for replying. Those who have responded with this particular error have stated that they’re using Edge 44 or Chrome.

I understand that its difficult to debug when the issue is consistently inconsistent. I’ve now sent a message to those who successfully completed the task asking for their browser/OS details, in case knowing what does work can shed some light on the situation.

Do you have any thoughts on the saving error?

Thanks,
Anthony

5

Hi @jon,

I’m a bit at a loss as to what to do here. I understand that it is hard to fix when you can’t replicate the problem. But I do need to move forward with my research. It seems that I may have to learn a new platform and move to that. Unless you can think of a better course of action? There are many, many reasons I don’t want to do start with another package…

6

Sorry, yes, looking at your study it’s certainly big and complex and I can certainly see places that we could optimize it to use many fewer opjects and that might well help.

The idea is basically to reduce the number of Routines and Components in your study and re-use them. E.g. you’ve got 4 almost-identical Routines for instructions, whereas you could have a single Routine (called instr_page) that received different text as part of a loop (called instr_pages). Similarly you’ve got a number of almost-identical routines for offers/bad_offers/good_offers/practice and more for

It won’t exactly fix the fact that there’s clearly a problem with some studies (possibly just when they get this big) but it might alleviate the issue in your case.

7

Interesting, thanks. It didn’t occur to me that just the number of routines/components would be a memory issue rather than the complexity of each. But even as typing that I realise that of course it would be more of a memory drain. I’ll work on optimising it better as you suggest.

One final question @jon… do you think this infinite “initialising experiment” is a related or separate issue? If separate, do you have a sense from issues raised previously as to which systems this occurs with more frequently? I’m not averse to stating that only certain system configurations can participate in the study.

Thanks again,
Anthony

8

I think the initialising experiment issue midway through the task is the result of webgl context_lost_webgl, which I think is an issue with resources. I had this issue running your task at the end of the first loop, but it is intermittent.

9

Thanks @dvbridges. Just to be clear, are you saying you had an initialisating experiment message after the task had started? I haven’t had reports of that before, only that the it froze on that page and the task never started?

So if this context_lost_webgl error is a resource issue, theoretically that could be solved by optimising the task better as per Jon’s message? Apologies if these are basic questions, my backend knowledge of this stuff is non-existent.

10

Yes, midway, but I have also had it at the beginning.

Yes it could be. I am not sure where the resourcing problem is, but you do have 150 conditions files, not sure if that would cause an issue. First step though, it to reduce the number of components and routines in your script. For example, in your practice routines for good and bad reputations, there are two separate loops. This could be reduced to one loop, for both good and bad practice groups. Another optimisation would be to reduce the number of text components used for your countdown. Currently you use 8 text components, each with their own number for the countdown, and this is for 3 separate routines (so 24 text components used for countdown). Instead, use one text component with a duration of 8 seconds, that updates every frame and add in the text box:

$Math.abs(Math.round(t-8))
11

Cool. I’ll look into optimising it more next week (am away at a conference atm). Thanks for the tips, I appreciate it.

12

Permanent ‘Initializing…’ is something I’ve only seen when using Edge, (or Internet Explorer previsouly but that’s now given a specific warning). It was caused by errors occuring that Edge didn’t flag properly as errors, so we couldn’t bring up any information about the problem. I’ve never seen a variant where it occured mid-study though

13

This was a slightly different “initialising study” page, with a sad face emoji, and I received this using Chrome. When I checked the console, I saw the web GL context error. But yes, also those issues with Edge could also be some of the errors being reported, or if people are using IE.

14

Here are the screens

errorConsole
errorConsole1920×1080 287 KB
errorScreen
errorScreen1920×1080 56.5 KB

15

Hi @dvbridges,

I was wondering if it has ever run properly on your machine? And so, how many times have you had to go through it to get these errors?

16

I have not run to the end because it is a long task, but this happened on the last run, although I did try to break it by running the task in several tabs to try and overload the browser. If you do the optimisations above and let me know when its complete, I will have a look if there is any more that can be done. If you need a hand doing that, just let me know.

17

Fantastic, thanks.

18

@dvbridges sorry to be a pain. I’ve made a few changes (more to come) but when I test it I think it’s still running the old version. To begin with I’d forgotten to switch it back to ‘Export HTML on Sync’ (I’d disabled this once I put it on prolific). But even now I’ve corrected that it doesn’t seem to be accessing the new HTML code when I click pilot.

Any thoughts?

19

@Anthony, yes it suggests that an error has occurred trying to compile the code, probably related to the code component. If you open Coder, and then try to resync from Builder, you should get an error output into the Coder output window, if there is an error in one of the the code components.

20

Hi @dvbridges,

I’m trying to optimise the task as @jon suggested by having the instruction text be read in from the a conditions file, thereby only needing one routine. But instead of the text I’m getting [object Object] written on the screen, and the task stays on this screen.

I’ve tried to get round this by foregoing the condition file and coding the contents oof the text box with .setText on each loop, but I’m being defeated by the fact that \n doesn’t seem to work to produce a new line. I’ve also tried \r but this doesn’t work either. When I do either of these, the code doesn’t compile and I get a error below in the output:

image
image1829×83 20.3 KB

If you know a way to make either of these methods work I’d be eternally grateful. The currently synced version is the attempt at using condition file again.

Cheers,
Anthony