Pilot vs Running mode

Online experiments Pavlovia Surveys
1

I wondered if there are any significant differences between piloting mode and running mode of experiments, in they way of how they are run in the background.

A friend of mine created surveys and then an experiment with these surveys and while the experiment runs without problems in pilot mode, it is unusable in running mode. The code is exactly the same, there is no difference in the code for piloting and running mode but somehow, when the experiment is run in running mode, some code is executed simultaneously, rendering two parts of the experiment on top of each other and making the experiment unusable.

The surveys are in running mode and creating a new experiment with the same code turns up the same bug. We are a bit at the end of our ropes here, we see no obvious mistakes in the code and Claude didn’t find anything either. It seems like the only possibility is that the background technologies between pilot and running mode are a bit different which leads to the code reacting differently.

Has anyone experienced anything like this before? Does anyone have any insights into the differences between pilot and running mode and what could cause a bug like this?

2

Are you piloting locally or in the browser? I know that embedded surveys download data during the experiment when piloting.

Please could you clarify what you mean by rendering two parts of the experiment on top of each other? Are they two different components?

3

The experiment is running online on Pavlovia and thats where the bug appears. I sadly did not manage to get the project running locally so I cannot see if the bug is there as well.

The experiment has multiple “parts”. First there is info and consent for the participant (part 1), then there is a survey (part 2) and after that comes a task where the participant has to click on things (part 3). Part 1 runs normally. Then, part 2 starts with a short information and a continue button. In running mode, this info display is displayed and at the same time part 3 is displayed. Clicking on the screen only advances in part 3. So there are literally two things happening at the same time and displayed on top of each other, you can see both texts overlapping and all. Pilot mode runs everything normal, one after another.

4

So you have an embedded survey that doesn’t disappear when the experiment continues?

Does your survey contain any redirecting code?

5

I dont think it has any redirecting code, I’ll have to check with the survey owner again but I haven’t seen anything.

It’s not so much that the survey doesn’t disapper, its more that the survey can’t be started at all in running mode. There is a short info window where you are supposed to click on “continue” to start the survey. But in running mode, a few miliseconds after this info-screen appears, the next step in the experiment, which is supposed to come after the survey, appears on top of this info screen, making it impossible to click on “continue” to start the survey. Instead, you click on this new overlay which then starts the next experiment step, but the info screen still never disappears.

6

What version of PsychoPy are you using?

I would check that the survey itself is in Running mode. You could also try saving it and using the JSON instead of the ID. I think that was necessary for one version over the past year.

7

We use version 2025.2.3. The surveys are in running mode, we already checked that. I will try to use the json instead of the ID today, thanks.

8

Please could you add me (Wake) as a developer to the project so I can witness and try to replicate the issue?

9

I added you to my project. I cant really tell you much about the structure as its not my project. I added the json files of the surveys my friend sent me, but I did not manage to use them instead of the survey IDs, when I did the project didnt run anymore (errors or just a white screen with browser console errors).

Maybe screenshots from the project itself:

In piloting mode we have both parts separate, first the one on the left and when this part is done the second one comes up.

image
image1794×532 78.8 KB

But if I put the project in running mode, I get this:

Overlayed
Overlayed2496×1268 164 KB

Thank you for your time and effort!

10

Hi I’m the friend that originally designed the experiment. And I did some more testing.

It seems like that the routine after a Pavlovia-Survey gets skipped before it is finished. This happens after every Pavlovia-Survey, not only after the consent.

E.g. if the flow is like this: Pavlovia_survey → scr_oci_check → TopRight

Using the console to print the routine start times, it will become apparent that TopRight starts way to early; it starts before scr_oci_check is finished (e.g. ROUTINE START: scr_oci_check 27.085, ROUTINE START: TopRight 27.107).

Thank your for your help

11

The Bug also happens when using version 2026.1.3

12

I note that your JS code sets capped to true or false but your Each Frame code checks for capped != None.

I think your shelf code needs “await”, unless the idea is that the routine lasts until the shelf code is processed. This might solve your issue.

Your have continueRoutine = False in End Routine code where it does nothing – the routine has already ended.

13

I’ve used “await” before, but using this lead to a similar bug in Pilot mode, where a routine got skipped in the middle of the experiment when shelf call returned. That’s why I used “.then”.

I’m no expert but the other surveys (participant_information and debriefing_completion) that produce the bug are not connected to shelf calls where I could use await I think.

But I seem to have found a “solution”. I put a routine with an empty dummy text with a duration of 1 second after each survey. Now everything seems to work.