Randomization of stimuli for each group in Pavlovia shelf

Online experiments
, ,
1

Hi everyone

I have a set of 300 images in total that I want to be rated by participants in my experiment. In order to keep down the duration of the experiment, I decided that each participant should rate only 100 of those.
Before rating the images, participants will be assigned to one of 13 groups according to their score in a screening questionnaire using the pavlovia shelf. For each of the groups I set a participation limit of 18.

Any suggestions on how can I randomize the 300 pictures within each of those groups and how I can make sure all pictures are rated equally often within each group?

Thank you in advance!

2

Hello @johanna

The simplest solution would be to present a subset of your stimuli based on the grouping. You could use the Selected Rows parameter of the loop.

Best wishes Jens

3

Hi Jens

Thank you for the answer and suggestion. I am not sure if I understand the functioning of selected rows correctly but would this not lead to “bundles” of stimuli that always stay the same? So it would not be fully randomized?

Thanks!
Johanna

4

How about a dictionary on the shelf containing a list of 300 zeros for each group name?

At the start of the experiment, download the shelf entry and shuffle a list of 300 indices.

Go through the shuffled list and add the first 100 indices that have a value below 6 are added to a second list and add 1 to the shelf dictionary. Use this second list as the useRows variable for selected rows.

Update the shelf entry with the updated dictionary.

I think this would be better than the method I suggest in my Presentation Cap online demo
PsychoPy Online Demos because that accesses the shelf too often. It would be better to accidentally present the pictures slightly unevenly than to have a slower presentation on every trial.

5

Do you mean a separate dictionary on the shelf for each of the 13 groups?
With each dictionary created as {“image001”: 0, “image002”: 0, …, “image300”: 0}?

So if a participant scores 5 in the screening questionnaire, he or she gets assigned to the “score5” group and I add a code component that loads only the dictionary “score5_randomization” in the shelf while those of the other groups remain the same?

Can I further update the shelf entry with the updated dictionary only at the very end of my experiment? With this I could make sure that the “score5_randomization” dictionary gets only updated when tthe participants has completed the full study and not by those who droped out mid experiment.

6

More like:

{"score1": [0,0,0,0,0,0,0,0,0,....0], "score2": [0,0,0,0,0,0,0,0,0,....0], etc.}

Correct

7

Hello @johanna

Yes, in its simplest form the content of each list would be the same for each group. You have to ask yourself whether complete randomisation across all participants, with equal presentation of each stimulus, is worth the effort and will properly control for confounding variables. Instead of a randomisation with even distribution of stimuli, you could construct three lists in which you control for the confounding variables.

Never forget that a random selection of stimuli could result in a list with undesirable properties. Randomness is not always the answer.

Having said that @wakecarter suggests a way worth trying.

Best wishes Jens

1 Like
8

Hi @wakecarter

I decided to go with your suggestion since this seems to be the only way to make sure every picture gets rated equally often within each group.

In theory, I understand what your suggestions aims for. However, I am not used to work with the pavlovia shelf (and I am not experienced with python) so I struggle to set this up.

For now I set up a dictionary on the shelf with 10 zeros for each of the 13 groups and I have an excel file with 10 images.

But I don’t understand how to connect these two in my experiment? Could you give me some advice on the code I will need for this?

Thank you in advance!

9

Here’s my tip on using the Pavlovia Shelf

At the start of the experiment you need to retrieve the list for the current group.

// Retrieve the value of a dictionary field in a Shelf entry
fieldValue = await psychoJS.shelf.getDictionaryFieldValue({key: ["shelfName"], fieldName:"fieldName", defaultValue:"fieldName not found"});

fieldValue in this case will be your list of 10 images

You will also need a list of image indices.

imageIndices = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
shuffles(imageIndices)
useRows = []
for Idx in range(10):
    if fieldValue[imageIndices[Idx]] < 5:
        fieldValue[imageIndices[Idx]] += 1
        useRows.append(imageIndices[Idx])
    if len(useRows) == 5:
        break

This should give you a list of rows to select which are a random choice from your ten images with a maximum of 5 presentations of each within each group.

Then update the shelf at the end of the experiment

// Update the value of a dictionary field in a Shelf entry
await psychoJS.shelf.setDictionaryFieldValue({key: ["shelfName"], fieldName: "fieldName", fieldValue :fieldValue});
10

Thank you! I tried it but now the experiment gets stuck on “initialising”. The browser console shows the error message “Uncaught SyntaxError: missing ( after for”. Where do I close that bracket again? Every position I put it just leads to another error message.

11

Hello

It is not clear from your description where the error might be. You may have missed a closing bracket, you may have an opening bracket too many, the order of brackets might be wrong [( instead of ([ or )] instead of ]). Have a look at Wakefield's Daily Tips - #5 by wakecarter to find out how to locate an error, or at least where the interpreter is going wrong.

Best wishes Jens