There are many complaints about the shelf for counterbalancing in online studies on the forum here. I saw them but decided to give it a try nevertheless, hoping that things became better since the first complaints were from 2024 or even earlier. But no: we had persistent issues with shelf locking out the study while it was not yet complete. Worst case, the participants got through the study but the data were not saved with the locked shelf error:
"Unfortunately we encountered the following error:
when confirming or cancelling a participant’s participation to the counterbalanced design with key: [“counterbalance_confb”,“@designer”,“@experiment”]
unable to access the record for the given key: either it does not exist, or it is locked, possibly by another experiment session
Try to run the experiment again. If the error persists, contact the experiment designer".
The shelf also does not free reserved spots automatically when the participant quits the study. So in its current form, it is not just unusable, it can be actively harmful as it creates the impression that everything works on initial attempts, but then locks out participants when the active data collection begins.
I understand that PsychoPy devs are a small group, and am very grateful for continuous open-source development, but I hope that removing the counterbalancing routine is easy and could be done quickly.
Please could you give me counterbalance shelf code a test, and see if that works for you?
I realise that isn’t what you are asking and I will suggest that we add a warning to the counterbalance routine for online users at our next planning meeting.
We are running a pilot and I’m now trying to get the remaining 4 subjects in one group and 3 in another (out of 4 groups). My plan is to use group as a url parameter and just make two copies on Pavlovia, it seems easier. But I’ll try your version next time. I saw it on some threads but was lazy and decided to stick with “official” routine, which, in hindsight, wasn’t the smartest thing to do =/
You should be able to reset the counterbalance routine (removing reserved) and tell it how many participants you want in each group for the rest of the study, though custom URLs will also work.
Yeah, I tried that first, removed all reserved, adjusted the numbers, but it continued adding subjects to existing groups even though the current N was >= max.