Shelf counterbalance uneven

Online experiments
1

I am currently running an experiment on Pavlovia and am using the new counterbalance routine connected to a shelf counterbalance record.

At the moment, the groups seem pretty unbalanced—10 in one group and 5 in the other. I’m not certain how the reserved slots are counted, but if it works as I suspect—that started runs get listed as reserved for some period of time—it seems even more unbalanced, with a total of 13 starts with one condition and only 6 for the other.

Should I expect that over a larger number of participants this will balance out? Or is there something I may have missed?

Screenshot 2025-08-21 at 9.03.01 PM
Screenshot 2025-08-21 at 9.03.01 PM436×386 6.14 KB

PsychoPy version: 2025.1.1

URL of experiment: Pavlovia

2

You have a target of 300 in each group. Is that split into chunks? If not then I’m not surprised about some unevenness near the start. Equally, the shelf doesn’t cope well if the chunks are too small.

3

Thanks @wakecarter . The 300 are not split into chunks, just 300 slots with nb reps of 1—I saw in previous threads that people were having better success with larger “slots” and lower “nb reps”.

Is there a preferred size for either slots or nb reps? (I’m a little unclear on what woukd differ in an experiment with say 10 slots x 3 nb reps vs 30 slots x 1 nb reps).

Also, if the experiment is already running, is it a bad idea to change the shelf entry to something like 100 slots x 3 nb reps?

Thanks!

4

Are you planning/likely to get 600 participants? If so, then you could probably leave it.

Otherwise I think it should be okay to change the criteria part way through. If you had 50 x 6 then that should balance the number of participants in each group every 100 (though you may need to manually delete the reserved slots if that isn’t happening automatically). Very small slots with a high number of repeats are discouraged and cause issues.

5

We’re planning on ~200 participants. I just set the counterbalance high so we didn’t hit a ceiling.

Perhaps changing it to something like 20x5 or 10x10 for each group?

6

10x10 might be too low. You could try 25 x 5 or 50 x 3 so you have some headroom

7

Thanks; I’ll give it a go!

8

FWIW, it still seems to be favoring one condition over the other by 5-10 participants.

I’ve tried a few variations, but it’s presently at 24 x 9; group 1 has 45 completed/22 reserved; group 2 has 37 completed/22 reserved.

9

You could try deleting the reserved ones to see if it evens out at 48/48

10

Thanks, I’ll try that.

How long do the reserved slots persist for, if one doesn’t remove them manually?

11

I think that they are supposed to expire after something like 1 hour or 24 hours, but that functionality doesn’t yet seem to be working.

1 Like