Error in a loop of images in online version only

Great. What sort of bugs are you experiencing?

I’ve nailed the problem down.

My task involves some images that can be clicked on to select and then another location can be clicked and the image will change location (like drag and drop but without the dragging - I’d used drag and drop but found it a little laggy sometimes).

This requires the basic properties in the builder for each image, in the line that reads “Image”, to point to the image, e.g. “image1.png”, and for the drop down box to be set to “set every frame”.

It is the setting in the drop down box that causes the problem, if it is set to constant, the images display both locally on PsychoPy and online through Pavlovia, but the images are not movable. If it is set to “set every frame”, when run on PsychPy locally, the images display and can be clicked and moved, but online, the PIXI error above occurs as soon as that routine begins.

Ok, yes that is how components tend to work. Setting to “Every repeat” will update the variable on every trial, whereas “Each Frame” will allow the variable to be updated on every frame, if required. http://www.psychopy.org/builder/components.html#how-often-to-evaluate-the-variable-code

I’d really like to use the PsychoJS export to HTML functionality to run this online, but is updating the images each frame (as is necessary for this experiment) currently incompatible with this? At the moment it always generates the PIXI error.

Do the images really need to change on every frame? From what you describe, isn’t it just the location that needs to vary dynamically?

That setting just ensures that the images are automatically redrawn in the new location if they are moved. If it is not set, then then even if an image’s location technically changes, it will not appear so to the user because the image is not drawn again in the new location.

The experiment works perfectly in normal PsychoPy, it’s just when I try to run it online after converting to Javascript that there is this error.

That indicates something is going deeply wrong: it is absolutely standard behaviour for a stimulus to be redrawn on every frame on which it is set to be visible. It certainly shouldn’t need to have its source image file re-imported: that in itself will lead to many problems.

i.e. updating the image file for a stimulus on every frame (even when you’re re-loading the same image file) is highly likely to lead to performance issues, as that is an operation that often can’t be completed within a single refresh period. Very rapid alteration of image files requires that the stimuli be pre-generated and cached for the time time-critical presentation period. This isn’t something that is done for routine stimulus operations, like just updating its position.

Ah OK, interesting, thank you. I’d made an assumption about that setting based on the behaviour I’d observed in this experiment that was wrong.

Leaving the Javascript version to one side for the moment, I’ve found that in PsychoPy the experiment simply doesn’t work if the images aren’t all set to update every frame (the images just don’t move). I’m using some simple additional code to select an image by mouse click and then to set a new location for the image with a second mouse click. When set to update the images every frame the experiment behaves as intended and the timings are acceptable - the important data in this case is the final spatial arrangement of the images.

What I would really like to do though, is to convert this experiment to run online using PsychoJS. So if some change can be made to how image updates are handled that preserves the outwardly appearing current behaviour, but means that it works in the Javascript version, that would be amazing.

I suspect there is a conflict introduced by that custom code, which is interfering with the normal drawing cycle. We’d need to see it to make a better diagnosis.

You were correct! Thank you very much for prompting me to look into this. I think the way I previously had it running was introducing a delay as a side effect, which made the code I wrote work correctly and prevented an image from being picked up and then set down again in the same mouse click. I’ve made changes to this which result in the experiment working fine in PsychoPy, but when I export to HTML and run via Pavlovia, the experiment now displays the images, but none can be clicked and moved in the intended way.

The clicking and dropping is now restricted using frame numbers, is this incompatible with PsychoJS?

For reference, I set up some variables with default values on the ‘Begin Routine’ tab:

stimuli = [image1, image2, image3, image4]
clicked = 0
clickedStimulus = -1
pickupFrame = -1
dropOffFrame = -1

and then have this code on the ‘Each Frame’ tab:

if clicked == 0 and dropOffFrame < frameN - 8:
    for i, stimulus in enumerate(stimuli):
        if mouse.isPressedIn(stimulus):
            clicked = 1
            clickedStimulus = stimulus
            stimulus.opacity = 0.5
            dropOffFrame = -1
            pickupFrame = frameN
            break


if clicked == 1 and pickupFrame < frameN - 8:
     if mouse.getPressed().count(1):
        clickedStimulus.setPos(mouse.getPos())
        clickedStimulus.opacity = 1
        clicked = 0
        clickedStimulus = -1
        pickupFrame = -1
        dropOffFrame = frameN

Glad you have gotten things sorted, in Python at least. I don’t know enough about the current state of PsychoJS to know if there is a limitation there on this sort of dynamic interaction. Perhaps @dvbridges or @jon could comment?

But to help, you should explain exactly what you mean by this:

Do you mean they can’t be moved at all, or they move in some unexpected fashion?

Sure @Michael, I have some code which can switch the position of an image if it is clicked. The image position will need resetting at the beginning of each routine. Is a code component with the JS code view:

//Begin Routine

image.setPos([0, 0]) // original position - note, position must be set in square brackets

// Every Frame

for (let blockName in mouse.clicked_image) {
    if (mouse.clicked_image[blockName] == image.image) {
        image.setPos([.5, 0]);  // Shift position if mouse clicks image
    }
}
1 Like

@Michael, I should have been clearer, what I meant to say was that the images all display, but none of them move at all.

@dvbridges, thanks for this code, but I can’t seem to get it to work. I’ve set up a new experiment with a single routine that contains only single ‘image’, ‘mouse’ and ‘code’ objects. It runs, but the image just remains static. Is there anything obvious that I might be missing here? If I can get this to work in the simplest case, I’m sure I can adapt it to my needs at a later point.

Two warnings are visible in Chrome DevTools, but as far as I can see these only relate to the lack of an image mask in the present case.

log4javascript.min.js:148 WARN ImageStim.setMask visual-3.0.0b9.js 221 | setting the mask of ImageStim: image with argument: undefined.
log4javascript.min.js:148 WARN ImageStim._setAttribute util-3.0.0b9.js 771 | setting the value of attribute: mask in PsychObject: image as: undefined

Ok this works for me, using an image defined using a conditions file. The image is called ‘image’, and mouse called ‘mouse’. Would you show me your JS code from your simple example?

Thanks again

/***************** 
 * Untitled Test *
 *****************/

import { PsychoJS } from './lib/core-3.0.0b9.js';
import * as core from './lib/core-3.0.0b9.js';
import { TrialHandler } from './lib/data-3.0.0b9.js';
import { Scheduler } from './lib/util-3.0.0b9.js';
import * as util from './lib/util-3.0.0b9.js';
import * as visual from './lib/visual-3.0.0b9.js';

// init psychoJS:
var psychoJS = new PsychoJS({
  debug: true
});

// open window:
psychoJS.openWindow({
  fullscr: true,
  color: new util.Color([0, 0, 0]),
  units: 'use prefs'
});

// store info about the experiment session:
let expName = 'untitled';  // from the Builder filename that created this script
let expInfo = {'participant': '', 'session': '001'};

// schedule the experiment:
psychoJS.schedule(psychoJS.gui.DlgFromDict({
  dictionary: expInfo,
  title: expName
}));

const flowScheduler = new Scheduler(psychoJS);
const dialogCancelScheduler = new Scheduler(psychoJS);
psychoJS.scheduleCondition(function() { return (psychoJS.gui.dialogComponent.button === 'OK'); }, flowScheduler, dialogCancelScheduler);

// flowScheduler gets run if the participants presses OK
flowScheduler.add(updateInfo); // add timeStamp
flowScheduler.add(experimentInit);
flowScheduler.add(trialRoutineBegin);
flowScheduler.add(trialRoutineEachFrame);
flowScheduler.add(trialRoutineEnd);
flowScheduler.add(quitPsychoJS);

// quit if user presses Cancel in dialog box:
dialogCancelScheduler.add(quitPsychoJS);

psychoJS.start({configURL: 'config.json', expInfo: expInfo});

var frameDur;
function updateInfo() {
  expInfo['date'] = util.MonotonicClock.getDateStr();  // add a simple timestamp
  expInfo['expName'] = expName;

  // store frame rate of monitor if we can measure it successfully
  expInfo['frameRate'] = psychoJS.window.getActualFrameRate();
  if (typeof expInfo['frameRate'] !== 'undefined')
    frameDur = 1.0/Math.round(expInfo['frameRate']);
  else
    frameDur = 1.0/60.0; // couldn't get a reliable measure so guess

  // add info from the URL:
  util.addInfoFromUrl(expInfo);

  return Scheduler.Event.NEXT;
}

var trialClock;
var image;
var mouse;
var globalClock;
var routineTimer;
function experimentInit() {
  // Initialize components for Routine "trial"
  trialClock = new util.Clock();
  image = new visual.ImageStim({
    win : psychoJS.window,
    name : 'image', 
    image : 'picture1.png', mask : undefined,
    ori : 0, pos : [0, 0], size : [0.5, 0.5],
    color : new util.Color ([1, 1, 1]), opacity : 1,
    flipHoriz : false, flipVert : false,
    texRes : 128, interpolate : true, depth : 0.0 
  });
  mouse = new core.Mouse({
    win: psychoJS.window,
  });
  mouse.mouseClock = new util.Clock();
  
  // Create some handy timers
  globalClock = new util.Clock();  // to track the time since experiment started
  routineTimer = new util.CountdownTimer();  // to track time remaining of each (non-slip) routine
  
  return Scheduler.Event.NEXT;
}

var t;
var frameN;
var gotValidClick;
var trialComponents;
function trialRoutineBegin() {
  //------Prepare to start Routine 'trial'-------
  t = 0;
  trialClock.reset(); // clock
  frameN = -1;
  // update component parameters for each repeat
  // setup some python lists for storing info about the mouse
  mouse.clicked_name = [];
  gotValidClick = false; // until a click is received
  image.setPos([0, 0])
  // keep track of which components have finished
  trialComponents = [];
  trialComponents.push(image);
  trialComponents.push(mouse);
  
  for (const thisComponent of trialComponents)
    if ('status' in thisComponent)
      thisComponent.status = PsychoJS.Status.NOT_STARTED;
  
  return Scheduler.Event.NEXT;
}

var continueRoutine;
function trialRoutineEachFrame() {
  //------Loop for each frame of Routine 'trial'-------
  let continueRoutine = true; // until we're told otherwise
  // get current time
  t = trialClock.getTime();
  frameN = frameN + 1;// number of completed frames (so 0 is the first frame)
  // update/draw components on each frame
  
  // *image* updates
  if (t >= 0.0 && image.status === PsychoJS.Status.NOT_STARTED) {
    // keep track of start time/frame for later
    image.tStart = t;  // (not accounting for frame time here)
    image.frameNStart = frameN;  // exact frame index
    image.setAutoDraw(true);
  }
  for (let blockName in mouse.clicked_image) {
      if (mouse.clicked_image[blockName] == image.image) {
          image.setPos([1, 1]);
      }
  }
  
  // check if the Routine should terminate
  if (!continueRoutine) {  // a component has requested a forced-end of Routine
    return Scheduler.Event.NEXT;
  }
  continueRoutine = false;// reverts to True if at least one component still running
  for (const thisComponent of trialComponents)
    if ('status' in thisComponent && thisComponent.status != PsychoJS.Status.FINISHED) {
      continueRoutine = true;
      break;
    }
  // check for quit (the Esc key)
  if (psychoJS.experiment.experimentEnded || psychoJS.eventManager.getKeys({keyList:['escape']}).length > 0) {
    psychoJS.quit('The [Escape] key was pressed. Goodbye!');
  }
  
  // refresh the screen if continuing
  if (continueRoutine) {
    return Scheduler.Event.FLIP_REPEAT;
  }
  else {
    return Scheduler.Event.NEXT;
  }
}


function trialRoutineEnd() {
  //------Ending Routine 'trial'-------
  for (const thisComponent of trialComponents) {
    if (typeof thisComponent.setAutoDraw === 'function') {
      thisComponent.setAutoDraw(false);
    }
  }
  // store data for thisExp (ExperimentHandler)
  
  // the Routine "trial" was not non-slip safe, so reset the non-slip timer
  routineTimer.reset();
  
  return Scheduler.Event.NEXT;
}


function endLoopIteration(thisTrial) {
  // ------Prepare for next entry------
  return function () {
    if (typeof thisTrial === 'undefined' || !('isTrials' in thisTrial) || thisTrial.isTrials) {
      psychoJS.experiment.nextEntry();
    }
  return Scheduler.Event.NEXT;
  };
}


function importTrialAttributes(thisTrial) {
  return function () {
    psychoJS.importAttributes(thisTrial);

    return Scheduler.Event.NEXT;
  };
}


function quitPsychoJS() {
  psychoJS.window.close();
  psychoJS.quit();

  return Scheduler.Event.QUIT;
}

Ok, not sure what is happening tbh. Try this example attached and see how they differ. For the png files, just make some quick colors images in paint and match them to the filenames in the cond sheet i.e., red.png etc.

cond.xlsx (7.8 KB)
imLoop.psyexp (7.1 KB)

@dvbridges, thanks for this. I ran the experiment you posted and the images move as expected. I’ll take a more thorough look at this and try to make it work for my needs, but this is super helpful.

Hi There,

I am experiencing a similiar problem as amr’s first question, with the error code:

  • when setting the autoDraw attribute of stimulus: image
  • the PIXI representation of the stimulus is unavailable

Psychopy version - 3.07
Windows version - 10; 64 bit

Description of the problem :

In the above solution it seemed this was fixed by changing the image to set every repeat. In my experiment the images are set to load during an ISI, as previously it was struggling to load fast enough.

If you could please advise any steps I might be able to take to resolve this error that’d be much appreciated.

Apologies, forgot to post link to experiment, please see below.

https://pavlovia.org/run/maxnni/csfp_1-05/html/

For anyone who finds this thread via search like I did.

I was also getting the error:

  • when setting the autoDraw attribute of stimulus: image
  • the PIXI representation of the stimulus is unavailable

I had recently updated my conditions .xlsx file to include rows which were missing, but was still getting this error.

I restarted psychopy and cleared my browser cache and the error resolved itself. Maybe the experiment was using an old version of the conditions file and clearing the cache forced it to to use the updated version.