My experiment works locally, but online I get different errors such as initialising the experiment and unknown resource errors

URL of experiment:

Description of the problem:
I have now tried to update a relatively simple experiment, but no matter what I do, I get different errors. I have been working on it all week, but nothing works online. Offline it works no problem.

Currently, it is stuck on “initialising the experiment”, previously I got “unknown resource errors” despite having manually added the files in “online” under the properties menu

The essential thing my experiment should do is to show 200 images, each either in its original state or run trough one of 3 types of distortion. Observers should then rate them 1 to 5. My local python version does this well, although I would still like to set a seed for each observer and other minor tweaks such as only downloading the necessary images.

Previously, one issue was that I imported the random package in python and that this didn’t work in JS, but I should now have addressed this. However, nothing seems to work in getting it online.

Thank you for reading through my ramblings and I would really appreciate anything that can help me forward. I had hoped to run this experiment this week and I cannot see how I can even run it next week without some help.

Hi Simon, the URL you posted gives a 404 error.

Hi Ajus. Thanks for trying to help me. Here’s the link again. Is there anything I have to do on my side to make it accessible? Sign in · GitLab

Still, does not work for me. I think you have to go to General Settings → Permissions → Visibility → Public.

1 Like

Thanks. I think I got it to work now. At least I can open it from incognito mode.

So, there were multiple problems with capitalization of files endings and directories. I fixed those and now the instructions run fine online. (see here: Sign in · GitLab). But then the trials begin and do not work. I think this is because A) the code is in the wrong place (End Experiment) and B) the auto-translation of python to js does not work for the things you wrote. However, if I understand your code correctly, it is pretty easy to do what you want to do using just two loops instead of all the code. If you want to go through the 200 pictures and present the 4 versions in a randomized order, just put two loops around your trial sequence. The outer loop has to be on “sequential” and use a conditions file that contains variable “a” with the numbers 1 to 200, the inner loop has to be on “random” and contain the variable “b” with the four endings that specify each file. Then you can use $a + b in the image component. If that is not what you intend to do, never mind:smiley: Then you have to figure out how to appropriately translate your code and put it in “Begin Experiment”.

Hope that helps!

Thank you for your reply. I do not seem to be able to access your gitlab, however.

There might well be an easier way to code the loop, but I want to be sure, I convey the goal.

An observer should see each image once per experiment. However, If I want to counterbalance the same observer (may or may not be needed), then the observer should only see the images in different states of distortion (here named a to d). Notice that each experiment has the same amount of a,b,c & d.

Here it is simplified. Each column could be one experiment if I only had 8 images. In the end, the observer would have seen all combinations:

image

I can do this with the python code provided that I set a seed with an observer ID. I am not sure that your suggestion would give the same output?

Also, the code was definitely supposed to run before. I put it into after as a part of debugging. Apparantly, I didn’t upload the reverted.

Hi, now you should be able to access it. Okay, I think I get what you want to do. So basically, a participant sees 200 images and for each image, it is randomized which version (distortion) he sees, but “a” to “d” appear the same amount of times? Then that is not easily done with loops as far as I can tell. But if you already debugged the code, then that’s no problem. Then, you can just insert it in this version in the right place and it should work.

1 Like

Thank you again for your help. Here is a new version: Simon Del Pin / SopaNew · GitLab

I have now, isolated the issue to my costum code.

The full code that gets the job done in python is:

# Which round of the experiment?
expround = 1 # Later rounds than 1 can be used for counter balancing individuals
repeated = 20 # Number of trials to repeat for meassuring correlation
numimg = 200 # Number of unique images - MUST BE DIVISIBLE BY FOUR

# Use observer as seed?
#random.seed('34ad') # Try something like str(participant)
foo = [0,1,2,3]
shuffle(foo)

# Seed for latin square
digit = foo[expround-1]

# A letter for each type of distortion
alphabet=['.JPG','_06_03.JPG','_07_03.JPG','_08_03.JPG']
letter=alphabet*int(numimg/4)
shuffle(letter)

# Use a latin square to switch letters 
if digit == 0:
    ls = [1,0,3,2]
if digit == 1:
    ls =[0,2,1,3]
if digit == 2:
    ls = [3,1,2,0]
if digit == 3:
    ls = [2,3,0,1]

# Switch letters with one of four sequenzes    
lettershuf=[]
for l in letter:
    if l=='.JPG':
        lettershuf.append(alphabet[ls[0]])
    if l=='_06_03.JPG':
        lettershuf.append(alphabet[ls[1]]) 
    if l=='_07_03.JPG':
        lettershuf.append(alphabet[ls[2]])
    if l=='_08_03.JPG':
        lettershuf.append(alphabet[ls[3]])
        
# Stims
stims = []
for n in range(0,numimg):
    stims.append(str(n+1)+lettershuf[n])
shuffle(stims)

# Append a certain number of trials for testing internal correlation
for n in range(numimg-repeated,numimg):
    stims.append(stims[n])

# Shuffle so the tests for correlation don't all come in the end
shuffle(stims)

When I as a sanity check set just two images:
stims = ['1_06_03.jpg','12.jpg']
…My program correctly displays them.

But as soon as I try to implement more of the code. It all goes to mayhem.

Even just using the first line:
expround = 1 # Later rounds than 1 can be used for counter balancing individuals

… is seemingly enough that my program is stuck at initialising.

I am obviously not a professional coder, but I can’t understand what the issue is. There are a few lines of code, yes, but they should all be quite basic?

You have to put the code in “Begin Experiment” not “Before Experiment”. Also, if the program is stuck at initialising, you can open the console (ctrl + shift + K in firefox) to see the error. In your case it was “Uncaught ReferenceError: assignment to undeclared variable expround”. But this goes away when you put the code into “Begin Experiment”.

1 Like

Regarding your code itself: Correct me if I miss something, but would it not be way easier to shuffle your 200 images, and shuffle an array containing a,b,c, and d each 50 times and use the two shuffled arrays to define the image file for each repetition? Would this not do what you want? Of, course it would not guarantee that each image is presented equally often in each version across participants, but this you can never make sure when you generate the list of stimuli on the spot.

1 Like

Thank you for the hint about the console. That can def. be useful for figuring out what is wrong.

Adding the code to “Begin Experiment” still does something wrong. It seems like the definition of the images in some way goes wrong when we are in JS.

image

I would guess that my code

# Stims
stims = []
for n in range(0,numimg):
    stims.append(str(n+1)+lettershuf[n])
shuffle(stims)

Somehow is not correctly translated in JS with:

stims = [];
for (var n, _pj_c = 0, _pj_a = util.range(0, numimg), _pj_b = _pj_a.length; (_pj_c < _pj_b); _pj_c += 1) {
    n = _pj_a[_pj_c];
    stims.push(((n + 1).toString() + lettershuf[n]));
}

This is merely a guess though, as I really don’t understand JS.

I am not sure if your suggested algorithm would allow for one observer to see all the images over several experiments? If it would, then I would be very happy to see an implementation. My code may easily be unnecessarily complicated.

Yes, I would guess that the code is not correctly translated. Can you explain what you mean by “over several experiments”? Does one participant complete the experiment multiple times?

Yes. Or at least, I would like for the option that one observer could run the experiment up to an additionally 3 times extra and thereby see all combinations exactly once. However, I am starting to be pressed for time to at least run some pilot data so I guess, I could temporarily live without it.

I think that is hard to do on pavlovia, because it can’t tell if or how often a participant already took part in an experiment.

If you want to get starting without this possibility, I would try something along the lines of:

// in "Begin Experiment"

// function for generating a sequence
function range(start, end) {
          return Array(end - start + 1).fill().map((_, idx) => start + idx)
      }

// function for shuffling
function shuffle(array) {
  var copy = [], n = array.length, i;

  // While there remain elements to shuffle…
  while (n) {

    // Pick a remaining element…
    i = Math.floor(Math.random() * array.length);

    // If not already shuffled, move it to the new array.
    if (i in array) {
      copy.push(array[i]);
      delete array[i];
      n--;
    }
  }

  return copy;
}

// generate image array
image = range(1, 200)

// generate distortion array
A = Array(50).fill('A.JPG')
B = Array(50).fill('B.JPG')
C = Array(50).fill('C.JPG')
D = Array(50).fill('D.JPG')

distortion = A.concat(B).concat(C).concat(D)

// shuffle both
image = shuffle(image)
distortion = shuffle(distortion)

// merge both arrays element by element to get the final stim array
stims = [image , distortion ].reduce((a, b) => a.map((v, i) => v + b[i]))

Then you can use stims just like you intended to before.

1 Like

Thanks for the suggestion. It took me a moment to figure out how to insert it as js code. But I think I figured it out now. However, there’s a new error:

Using the console code, it seems that the underlying code that causes the error is:
image.setImage(("images/" + stims[trials.thisN].toString()));

Which I believe in turn comes from the python code defined in builder as:
$'images/' + str(stims[trials.thisN])

If this is the code that makes an error, I am slightly confused as I have run very similar code online before.

As for using observer as a random seed, I plan to run my experiment on prolific where each observer has a unique code. However, this is obviously secondary as long as I cannot display images at all.

Thank you so much for your help again.

Edit: And of course a link to the new version Simon Del Pin / SopaJsShuffle · GitLab

Glad to help! I think the problem now is, that I named the array in the beginning “image” and your component is also called “image”. If you just rename the image array in the code to something more unique this should work.

1 Like

After changing your code as suggested, I got a version that worked for a first pilot. After looking at the data and considering the difficulties with translating my python code into JS, I found a minimum that I hoped would work. Alas, it only works locally and not when auto-translated into JS.

The logic is:
Divide my images into 4 equal chunks, (e.g. 1:16,17:32…)
Give each chunk exactly the same number of distortions (‘.JPG’,‘_06_03.JPG’,‘_07_03.JPG’,‘_08_03.JPG’)
Create distortions for each chunk
Shuffle each and combine them
Finally, combine these with the numbers and shuffle the list.

It sounds clunky, but I can do it in very few lines in R:

# Take list (with 4 equal chunks, e.g. 1:16,17:32...)
# Give each chunk the same number of a,b,c,d
imgs <- 1:64 # Must be divisible by 16

# Create letters for one chuncks
let <- rep(letters[1:4],max(imgs)/16)
# Combine 4 chunks, each randomised
test <- c(sample(let),sample(let),sample(let),sample(let))

# Check that each chunck has the same number of letters
test <- as.factor(test) # Factor as overview, not necessary
summary(test[1:16])
summary(test[17:32])

# Combine and shuffle
images <- cbind(imgs,test)
images <- images[sample(1:nrow(images)), ]

My python code does the same thing, but less elegant (I have tried to keep it basic in the hope of the auto-translation getting it)

numimg = 192 # Number of unique images - MUST BE DIVISIBLE BY 16

# Define distortions
dist=['.JPG','_06_03.JPG','_07_03.JPG','_08_03.JPG']
letter1=dist*int(numimg/16)
letter2=dist*int(numimg/16)
letter3=dist*int(numimg/16)
letter4=dist*int(numimg/16)

# Shuffle each
shuffle(letter1)
shuffle(letter2)
shuffle(letter3)
shuffle(letter4)
# Combine
letter = letter1+letter2+letter3+letter4

# Stims
stims = []
for n in range(0,numimg):
    stims.append(str(n+1)+letter[n])
shuffle(stims)

This is auto-translated into:

numimg = 192;
dist = [".JPG", "_06_03.JPG", "_07_03.JPG", "_08_03.JPG"];
letter1 = (dist * Number.parseInt((numimg / 16)));
letter2 = (dist * Number.parseInt((numimg / 16)));
letter3 = (dist * Number.parseInt((numimg / 16)));
letter4 = (dist * Number.parseInt((numimg / 16)));
util.shuffle(letter1);
util.shuffle(letter2);
util.shuffle(letter3);
util.shuffle(letter4);
letter = (((letter1 + letter2) + letter3) + letter4);
stims = [];
for (var n, _pj_c = 0, _pj_a = util.range(0, numimg), _pj_b = _pj_a.length; (_pj_c < _pj_b); _pj_c += 1) {
    n = _pj_a[_pj_c];
    stims.push(((n + 1).toString() + letter[n]));
}
util.shuffle(stims);

Which gives the following error:

So it seems that something goes wrong with how the image is defined in JS. Again, it works fine in Python. Can you help me once again?

Thanks for reading trough this ramble :slight_smile:

Hi @simon.dp, I have to be missing something. What exactly is the point of this clustering? Are the images in a specific order such that it is important that the distortions are equally likely for each image in the cluster?

I guess your python code is already too advanced to be translated. You are probably better of just googling how to do something in js instead of writing it in python and counting on the translation. You could also just adapt my previous js code to your demands I guess. Even if I don’t really see the point yet :smiley:

Thank you for your response. I should probably have made it clearer but my image set consists of categories with 25% images of people, 25% images of animals and so forth. Every observer should see each image exactly once. Either in its original form or in 1 of 3 distortions.

My very first idea was to set a seed and make sure that each image was displayed with the same number of distortions throughout the experiment. This would be the ideal, but less will probably do. With the code I have posted above, I make it so that each observer has the same ratio of distortions and original images for each of the categories.

I had hoped it would be relatively trivial and I could write it out pretty easily in both R and Python. However, I have no experience with JS. It may, however, be worth it to get a JS development environment and test the code there rather than to try to run the experiment and download 800 MB each time just to get an error. (I actually have as a next step to download the images sequentially, but must prioritise that the experiment works first).