Screen freezes for most of the subjects and many data are empty

[intouman]

URL of experiment:
https://run.pavlovia.org/intouman/hse/html

Description of the problem:

We are running our behavioural pilot on Pavlovia and most of our participants are complaining about technical problems, with the most common one being that the screen freezes at some random point and the experiment crashes. We have received several angry reviews so far and even those subjects that managed to go through the whole experiment, reported that they, too, faced problems (statistics of the latter attached).

In addition, most of the data produced so far are appropriate, but there are more than 20 empty data in the data folder in gitlab, for which no credits have been consumed. As far as I know, if someone doesn’t complete the experiment, no data are produced. So, how come these data have been created?

We need participants with certain rare qualifications and now that we managed to find some, it’s a pity to lose them due to technical problems.

What we have done to solve it:

After reading previous posts here, we have instructed participants to use Google Chrome or Firefox to ensure that the problems are not due to the browser they are using. Also, we tried to minimize the code of the experiment, which is relatively long, but the best we could do was to have a 114kB js file.

Other characteristics of the experiment:

The experiment includes some big audio files (~39mB in total), but they are necessary.
I’m sorry, but the experiment is not in English. So, if you kindly decide to run it, please read the brief instructions in English first: Report - Google Docs. Overall, remember that you mouse click to move on and press SPACE or ENTER to the balloon task.

[wakecarter]

It sounds like memory errors. Do you have any Each Frame code updating objects even when they haven’t changed?

[intouman]

Thank you very much for the reply, @wakecarter. Yes, I’m attaching a screenshot of the JS Each Frame tab of my most complex code component.

Explanation:
When a stimulus appears on the screen, we want to prevent any responses for the first 2 seconds. The “Timer” clock controls exactly that and the “dep” variable just distinguishes the first response of the trial from the next responses (dep=0 at begin routine).

[wakecarter]

I’m assuming that nPumps gets translated into change on screen. Is that in a component?

[intouman]

Indirectly, yes. “nPumps” changes another variable, called “BalloonSize” (which is in a separate code component),

and subsequently, the “BalloonSize” variable changes the stimulus on screen (Image stimulus properties below)

[wakecarter]

I’m not sure how intelligent the components are but I’ve been avoiding “set every frame”

Instead you could have in a code component:

if balloonSize != oldSize:
     balloonBody.setPos([0,balloonSize/2-.5])
     balloonBody.setSize(balloonSize)
     oldSize = balloonSize

[intouman]

Thank you, I’ll try. I assume you mean to add those lines of code in the Each Frame tab of a code component, and in the image object to have both position and size in “set every repeat” mode.

[wakecarter]

Correct

[intouman]

Any ideas about the empty data that didn’t consume credits? Could it be that these actually correspond to completed experiments? That would be a bad scenario, partly because we can’t see the amount of money they have won and expect from us.

[wakecarter]

I guess it could be if they completed the experiment but didn’t wait for the “Thank you for your patience” message.

I couldn’t open the instructions in English so I just wanted to check that it was intentional that the return key was disabled for some balloons.

Also – using the console I could see the maximum number of pumps allowed and whether return was disabled. This might be important to note for experiments which give participants real money based on their performance.

[intouman]

Thank you for trying to run it. That makes sense. Yes, the RETURN is not allowed for blue balloons.

Oh thanks for the important note, could you please tell us how to hide that information?

[wakecarter]

When viewing the console I assume I was seeing the lines from Excel. Instead of having a column called maxpumps you could have a column called trialtype and have a separate non-linear lookup table (i.e. a dictionary or array) to convert trialtype into maxpumps. Alternatively, maxpumps could be generated randomly within the experiment.

[intouman]

Great, thank you again!

[intouman]

In terms of the empty data, it seems that in some cases the reason is not that they didn’t wait. A participant sent us the following screenshot. Does this look familiar?

[wakecarter]

What is the filename it is trying to create?
Are you using database or CSV options?
I’m guessing that it isn’t anything strange about characters entered into the expInfo box, since that’s an email address.

[intouman]

This is a sample filename of an empty csv: PARTICIPANT_hse_2020-06-16_21h49.22.541
The filenames of the data that are actually appropriate are identical with the above one.
We’re using CSV saving format.

[wakecarter]

Are you doing anything in code components related to the data file? Is any data being saved that might include special characters?

[intouman]

• Regarding the first question, all the variables I want to save are saved with this command:

• Regarding the second question, I can only consider the following related to your question:

1. One variable being saved is a character that includes commas, e.g. “23,23,25”

2. There are some variables coming from keyboard responses from multiple choice questions, which appear as [“4”] in the corresponding cell in the csv.

3. Another variable is saved with a name that includes spaces:

[wakecarter]

The commas might be the issue, but I’d expect it to work or not work rather than intermittently.