Load movie in an ISI

OS : Win10
PsychoPy version : v3.1.2
Standard Standalone? (y/n) : yes
What are you trying to achieve?: Load movie in an ISI period

What specifically went wrong when you tried that?: As part of a bigger experiment, we are trying to load movies in an ISI period but for some reason it doesn’t work. I’ve recreated the issue in a simpler scenario as below.

  • ISI period is 0.5 ms
  • Movie stimulus is from 0.5 ms to 1.5 ms
  • The movie file is specified as shown and set during the ISI period.

However, an error is generated to the effect of,

Traceback (most recent call last):
File “C:\Users\psychlab\Desktop\Testing\untitled.py”, line 115, in
depth=-1.0,
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py”, line 135, in init
self.loadMovie(self.filename)
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py”, line 182, in loadMovie
self._mov = VideoFileClip(filename, audio=(1 - self.noAudio))
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\moviepy\video\io\VideoFileClip.py”, line 91, in init
fps_source=fps_source)
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\moviepy\video\io\ffmpeg_reader.py”, line 33, in init
fps_source)
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\moviepy\video\io\ffmpeg_reader.py”, line 256, in ffmpeg_parse_infos
proc = sp.Popen(cmd, **popen_params)
File “C:\Program Files (x86)\PsychoPy3\lib\subprocess.py”, line 596, in init
_cleanup()
File “C:\Program Files (x86)\PsychoPy3\lib\subprocess.py”, line 205, in _cleanup
res = inst._internal_poll(_deadstate=sys.maxsize)
File “C:\Program Files (x86)\PsychoPy3\lib\subprocess.py”, line 1035, in _internal_poll
if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
OSError: [WinError 6] The handle is invalid
4.7101 WARNING User requested fullscreen with size [1920 1200], but screen is actually [1920, 1080]. Using actual size
6.5117 WARNING t of last frame was 12.31ms (=1/81)
Exception ignored in: <bound method MovieStim3.del of <psychopy.visual.movie3.MovieStim3 object at 0x0000027680268D68>>
Traceback (most recent call last):
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py”, line 495, in del
self._unload()
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py”, line 473, in _unload
self.clearTextures()
File “C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\visual\basevisual.py”, line 1003, in clearTextures
GL.glDeleteTextures(1, self._texID)
OSError: exception: access violation reading 0x0000000000000000

Lastly, when the movie is ‘set every repeat’, then it works fine.

In your case there isn’t really an advantage to using the preloading option. You can set it to load on “every repeat” and it will do the same thing as what you want anyway (start loading at t=0). With loading during the static period you would additionally get a warning if it took too long, but that isn’t likely given you have 500ms to load the movie. I suggest you simply delete the static period.

That said, I will also look into why it came up with an error. I can’t imagine what the problem would be off the top of my head.

Hi Jon,

Thanks for your reply. I’ll share a bit more detail of what the purpose of loading the movie in the static period was as I think there might be another issue in here. So our experiment looks like below,

  • BiasImageCue and GridStimulus are called upon by variables in a conditions file as linked in the loop.

All the timings seemed to be correct except for the 0.75s duration of the BiasImageCue (as below),


We then read a few forums and learnt of the static component and thought it’d be useful to preload the image only by setting it to load during the static component (note, the movie was still set to ‘every repeat’) and implemented the routine as below,


Note, that the movie component is disabled in the screenshot above as what we observed when it was enabled was as below,

The duration of the BiasImageCue was still not 0.75s and the static component started, not at time 0, but at some interval after that. Interestingly, adding the duration of what the BiasImageCue was displayed for and the ISI delay summed to what the correct duration of BiasImageCue should have been.

For completion, when the movie component was disabled in the above routine with the static component, the BiasImageCue was displayed for the correct duration.


All this lead us to try and load the movie during the static component as well to see if that resolved the timing issues but that was giving us different errors (one of them being the one in the original post).

We have worked around it by introducing a larger gap of 0.75 seconds (i.e., not as a static component) and that gives us the stimuli durations as expected but I am still curious about the delays observed with the static component.

I think I can explain most of these things:

Case 1 (top): trying to start and load all at t=0
Yes, you see erroneous timing here because the movie set to update every repeat is causing a delay at t=0 (i.e. the first frame of the Routine). That delays the drawing of the stimulus and the output on the p_port because the screen isn’t refreshed while the movie loads and that’s needed for the image to appear and get timestamped. The duration of the image will then be wrong if it starts late but stops at the correct time. You could fix that issue by setting it to last a fixed number of frames, for example. Then it will start late but have the specified duration.

Case 2: using the static comp for 0.5 s
Moving everything forwards by 0.5 s is probably sufficient to address the issues (assuming that the movie and image loads in less than 500ms). I’m not actually sure what went wrong here in terms of the timing of the static interval and cue. I’ll have to look carefully at the order in which the code is being written. Specifically I need to check that the static interval is written before other components. For other components the order of execution is clear, but for the static it isn’t (because of the way it’s rendered in the routine). I wonder if the issue is that the static is executed in order that it was added and therefore maybe it gets executed after the movie loading. That would lead to it being delayed and then it forcing everything else to run late. :frowning:

Case 3: using a delay but no Static
Glad this is working. I think in most cases it’s a reasonable approach. I’d like to check/fix Case 2 because that’s what I feel is the right way for this to work, but if that isn’t working right then your solution is totally reasonable.

thanks for a detailed report for us to troubleshoot. :slightly_smiling_face:

Hello again Jon, no worries at all. We use PsychoPy extensively and the PsychoPy community is always helpful, so happy to help whenever the opportunity arises :slight_smile:

So, I did a bit more troubleshooting and have 2 key things to highlight that might help you troubleshoot more specifically,

  1. Movie component and when it is set

So, to the working version (0.75 s gap, no static component, and both, image and movie, set to every repeat), all I added was a) a static component for the same duration and, b) set both, image and movie, to load during the ISI period. This is as below,

Interestingly, when I compiled the script for this, I noticed an anomaly. The ‘trial’ component of the script is below,

for thisBlock1 in Block1:
    currentLoop = Block1
    # abbreviate parameter names if possible (e.g. rgb = thisBlock1.rgb)
    if thisBlock1 != None:
        for paramName in thisBlock1:
            exec('{} = thisBlock1[paramName]'.format(paramName))
    
    # ------Prepare to start Routine "trial"-------
    t = 0
    trialClock.reset()  # clock
    frameN = -1
    continueRoutine = True
    routineTimer.add(4.000000)
    # update component parameters for each repeat
    GridStimulus = visual.MovieStim3(
        win=win, name='GridStimulus',
        noAudio = True,
        filename=StimulusMovie,
        ori=0, pos=[0, 0], opacity=1,
        loop=False,
        depth=-2.0,
        )
    Resp = keyboard.Keyboard()
    # keep track of which components have finished
    trialComponents = [ISI, BiasImageCue, GridStimulus, Resp, p_port]
    for thisComponent in trialComponents:
        thisComponent.tStart = None
        thisComponent.tStop = None
        thisComponent.tStartRefresh = None
        thisComponent.tStopRefresh = None
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
    
    # -------Start Routine "trial"-------
    while continueRoutine and routineTimer.getTime() > 0:
        # get current time
        t = trialClock.getTime()
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        
        # *BiasImageCue* updates
        if t >= 0.75 and BiasImageCue.status == NOT_STARTED:
            # keep track of start time/frame for later
            BiasImageCue.tStart = t  # not accounting for scr refresh
            BiasImageCue.frameNStart = frameN  # exact frame index
            win.timeOnFlip(BiasImageCue, 'tStartRefresh')  # time at next scr refresh
            BiasImageCue.setAutoDraw(True)
        frameRemains = 0.75 + .75- win.monitorFramePeriod * 0.75  # most of one frame period left
        if BiasImageCue.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            BiasImageCue.tStop = t  # not accounting for scr refresh
            BiasImageCue.frameNStop = frameN  # exact frame index
            win.timeOnFlip(BiasImageCue, 'tStopRefresh')  # time at next scr refresh
            BiasImageCue.setAutoDraw(False)
        
        # *GridStimulus* updates
        if t >= 2 and GridStimulus.status == NOT_STARTED:
            # keep track of start time/frame for later
            GridStimulus.tStart = t  # not accounting for scr refresh
            GridStimulus.frameNStart = frameN  # exact frame index
            win.timeOnFlip(GridStimulus, 'tStartRefresh')  # time at next scr refresh
            GridStimulus.setAutoDraw(True)
        frameRemains = 4 - win.monitorFramePeriod * 0.75  # most of one frame period left
        if GridStimulus.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            GridStimulus.tStop = t  # not accounting for scr refresh
            GridStimulus.frameNStop = frameN  # exact frame index
            win.timeOnFlip(GridStimulus, 'tStopRefresh')  # time at next scr refresh
            GridStimulus.setAutoDraw(False)
        if GridStimulus.status == FINISHED:  # force-end the routine
            continueRoutine = False
        
        # *Resp* updates
        if t >= 2 and Resp.status == NOT_STARTED:
            # keep track of start time/frame for later
            Resp.tStart = t  # not accounting for scr refresh
            Resp.frameNStart = frameN  # exact frame index
            win.timeOnFlip(Resp, 'tStartRefresh')  # time at next scr refresh
            Resp.status = STARTED
            # keyboard checking is just starting
            win.callOnFlip(Resp.clock.reset)  # t=0 on next screen flip
            Resp.clearEvents(eventType='keyboard')
        frameRemains = 4 - win.monitorFramePeriod * 0.75  # most of one frame period left
        if Resp.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            Resp.tStop = t  # not accounting for scr refresh
            Resp.frameNStop = frameN  # exact frame index
            win.timeOnFlip(Resp, 'tStopRefresh')  # time at next scr refresh
            Resp.status = FINISHED
        if Resp.status == STARTED:
            theseKeys = Resp.getKeys(keyList=['g', 'k', 'None'], waitRelease=False)
            if len(theseKeys):
                theseKeys = theseKeys[0]  # at least one key was pressed
                
                # check for quit:
                if "escape" == theseKeys:
                    endExpNow = True
                if Resp.keys == []:  # then this was the first keypress
                    Resp.keys = theseKeys.name  # just the first key pressed
                    Resp.rt = theseKeys.rt
                    # was this 'correct'?
                    if (Resp.keys == str(CorrAns)) or (Resp.keys == CorrAns):
                        Resp.corr = 1
                    else:
                        Resp.corr = 0
                    # a response ends the routine
                    continueRoutine = False
        # *p_port* updates
        if t >= 0.75 and p_port.status == NOT_STARTED:
            # keep track of start time/frame for later
            p_port.tStart = t  # not accounting for scr refresh
            p_port.frameNStart = frameN  # exact frame index
            win.timeOnFlip(p_port, 'tStartRefresh')  # time at next scr refresh
            p_port.status = STARTED
            win.callOnFlip(p_port.setData, int(1))
        frameRemains = 0.75 + .1- win.monitorFramePeriod * 0.75  # most of one frame period left
        if p_port.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            p_port.tStop = t  # not accounting for scr refresh
            p_port.frameNStop = frameN  # exact frame index
            win.timeOnFlip(p_port, 'tStopRefresh')  # time at next scr refresh
            p_port.status = FINISHED
            win.callOnFlip(p_port.setData, int(0))
        # *ISI* period
        if t >= 0 and ISI.status == NOT_STARTED:
            # keep track of start time/frame for later
            ISI.tStart = t  # not accounting for scr refresh
            ISI.frameNStart = frameN  # exact frame index
            win.timeOnFlip(ISI, 'tStartRefresh')  # time at next scr refresh
            ISI.start(0.75)
        elif ISI.status == STARTED:  # one frame should pass before updating params and completing
            # updating other components during *ISI*
            BiasImageCue.setImage(BiasCue)
            GridStimulus.setMovie(StimulusMovie)
            # component updates done
            ISI.complete()  # finish the static period
        
        # check for quit (typically the Esc key)
        if endExpNow or defaultKeyboard.getKeys(keyList=["escape"]):
            core.quit()
        
        # check if all components have finished
        if not continueRoutine:  # a component has requested a forced-end of Routine
            break
        continueRoutine = False  # will revert to True if at least one component still running
        for thisComponent in trialComponents:
            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                continueRoutine = True
                break  # at least one component has not yet finished
        
        # refresh the screen
        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen
            win.flip()
    
    # -------Ending Routine "trial"-------
    for thisComponent in trialComponents:
        if hasattr(thisComponent, "setAutoDraw"):
            thisComponent.setAutoDraw(False)
    Block1.addData('ISI.started', ISI.tStart)
    Block1.addData('ISI.stopped', ISI.tStop)
    Block1.addData('BiasImageCue.started', BiasImageCue.tStartRefresh)
    Block1.addData('BiasImageCue.stopped', BiasImageCue.tStopRefresh)
    Block1.addData('GridStimulus.started', GridStimulus.tStartRefresh)
    Block1.addData('GridStimulus.stopped', GridStimulus.tStopRefresh)
    # check responses
    if Resp.keys in ['', [], None]:  # No response was made
        Resp.keys = None
        # was no response the correct answer?!
        if str(CorrAns).lower() == 'none':
           Resp.corr = 1;  # correct non-response
        else:
           Resp.corr = 0;  # failed to respond (incorrectly)
    # store data for Block1 (TrialHandler)
    Block1.addData('Resp.keys',Resp.keys)
    Block1.addData('Resp.corr', Resp.corr)
    if Resp.keys != None:  # we had a response
        Block1.addData('Resp.rt', Resp.rt)
    Block1.addData('Resp.started', Resp.tStartRefresh)
    Block1.addData('Resp.stopped', Resp.tStopRefresh)
    if p_port.status == STARTED:
        win.callOnFlip(p_port.setData, int(0))
    Block1.addData('p_port.started', p_port.tStartRefresh)
    Block1.addData('p_port.stopped', p_port.tStopRefresh)
    

The movie component (i.e., GridStimulus) is under the comment of ‘update component parameters for each repeat’ and not initialised early on in the script as the image component is (i.e., BiasImageCue)

Moreover, when I set the movie component to ‘during repeat’ and compile the script, as below, it looks similar to the one compiled above (i.e., set during ISI),

for thisBlock1 in Block1:
    currentLoop = Block1
    # abbreviate parameter names if possible (e.g. rgb = thisBlock1.rgb)
    if thisBlock1 != None:
        for paramName in thisBlock1:
            exec('{} = thisBlock1[paramName]'.format(paramName))
    
    # ------Prepare to start Routine "trial"-------
    t = 0
    trialClock.reset()  # clock
    frameN = -1
    continueRoutine = True
    routineTimer.add(4.000000)
    # update component parameters for each repeat
    GridStimulus = visual.MovieStim3(
        win=win, name='GridStimulus',
        noAudio = True,
        filename=StimulusMovie,
        ori=0, pos=[0, 0], opacity=1,
        loop=False,
        depth=-2.0,
        )
    Resp = keyboard.Keyboard()
    # keep track of which components have finished
    trialComponents = [ISI, BiasImageCue, GridStimulus, Resp, p_port]
    for thisComponent in trialComponents:
        thisComponent.tStart = None
        thisComponent.tStop = None
        thisComponent.tStartRefresh = None
        thisComponent.tStopRefresh = None
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
    
    # -------Start Routine "trial"-------
    while continueRoutine and routineTimer.getTime() > 0:
        # get current time
        t = trialClock.getTime()
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        
        # *BiasImageCue* updates
        if t >= 0.75 and BiasImageCue.status == NOT_STARTED:
            # keep track of start time/frame for later
            BiasImageCue.tStart = t  # not accounting for scr refresh
            BiasImageCue.frameNStart = frameN  # exact frame index
            win.timeOnFlip(BiasImageCue, 'tStartRefresh')  # time at next scr refresh
            BiasImageCue.setAutoDraw(True)
        frameRemains = 0.75 + .75- win.monitorFramePeriod * 0.75  # most of one frame period left
        if BiasImageCue.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            BiasImageCue.tStop = t  # not accounting for scr refresh
            BiasImageCue.frameNStop = frameN  # exact frame index
            win.timeOnFlip(BiasImageCue, 'tStopRefresh')  # time at next scr refresh
            BiasImageCue.setAutoDraw(False)
        
        # *GridStimulus* updates
        if t >= 2 and GridStimulus.status == NOT_STARTED:
            # keep track of start time/frame for later
            GridStimulus.tStart = t  # not accounting for scr refresh
            GridStimulus.frameNStart = frameN  # exact frame index
            win.timeOnFlip(GridStimulus, 'tStartRefresh')  # time at next scr refresh
            GridStimulus.setAutoDraw(True)
        frameRemains = 4 - win.monitorFramePeriod * 0.75  # most of one frame period left
        if GridStimulus.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            GridStimulus.tStop = t  # not accounting for scr refresh
            GridStimulus.frameNStop = frameN  # exact frame index
            win.timeOnFlip(GridStimulus, 'tStopRefresh')  # time at next scr refresh
            GridStimulus.setAutoDraw(False)
        if GridStimulus.status == FINISHED:  # force-end the routine
            continueRoutine = False
        
        # *Resp* updates
        if t >= 2 and Resp.status == NOT_STARTED:
            # keep track of start time/frame for later
            Resp.tStart = t  # not accounting for scr refresh
            Resp.frameNStart = frameN  # exact frame index
            win.timeOnFlip(Resp, 'tStartRefresh')  # time at next scr refresh
            Resp.status = STARTED
            # keyboard checking is just starting
            win.callOnFlip(Resp.clock.reset)  # t=0 on next screen flip
            Resp.clearEvents(eventType='keyboard')
        frameRemains = 4 - win.monitorFramePeriod * 0.75  # most of one frame period left
        if Resp.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            Resp.tStop = t  # not accounting for scr refresh
            Resp.frameNStop = frameN  # exact frame index
            win.timeOnFlip(Resp, 'tStopRefresh')  # time at next scr refresh
            Resp.status = FINISHED
        if Resp.status == STARTED:
            theseKeys = Resp.getKeys(keyList=['g', 'k', 'None'], waitRelease=False)
            if len(theseKeys):
                theseKeys = theseKeys[0]  # at least one key was pressed
                
                # check for quit:
                if "escape" == theseKeys:
                    endExpNow = True
                if Resp.keys == []:  # then this was the first keypress
                    Resp.keys = theseKeys.name  # just the first key pressed
                    Resp.rt = theseKeys.rt
                    # was this 'correct'?
                    if (Resp.keys == str(CorrAns)) or (Resp.keys == CorrAns):
                        Resp.corr = 1
                    else:
                        Resp.corr = 0
                    # a response ends the routine
                    continueRoutine = False
        # *p_port* updates
        if t >= 0.75 and p_port.status == NOT_STARTED:
            # keep track of start time/frame for later
            p_port.tStart = t  # not accounting for scr refresh
            p_port.frameNStart = frameN  # exact frame index
            win.timeOnFlip(p_port, 'tStartRefresh')  # time at next scr refresh
            p_port.status = STARTED
            win.callOnFlip(p_port.setData, int(1))
        frameRemains = 0.75 + .1- win.monitorFramePeriod * 0.75  # most of one frame period left
        if p_port.status == STARTED and t >= frameRemains:
            # keep track of stop time/frame for later
            p_port.tStop = t  # not accounting for scr refresh
            p_port.frameNStop = frameN  # exact frame index
            win.timeOnFlip(p_port, 'tStopRefresh')  # time at next scr refresh
            p_port.status = FINISHED
            win.callOnFlip(p_port.setData, int(0))
        # *ISI* period
        if t >= 0 and ISI.status == NOT_STARTED:
            # keep track of start time/frame for later
            ISI.tStart = t  # not accounting for scr refresh
            ISI.frameNStart = frameN  # exact frame index
            win.timeOnFlip(ISI, 'tStartRefresh')  # time at next scr refresh
            ISI.start(0.75)
        elif ISI.status == STARTED:  # one frame should pass before updating params and completing
            # updating other components during *ISI*
            BiasImageCue.setImage(BiasCue)
            # component updates done
            ISI.complete()  # finish the static period
        
        # check for quit (typically the Esc key)
        if endExpNow or defaultKeyboard.getKeys(keyList=["escape"]):
            core.quit()
        
        # check if all components have finished
        if not continueRoutine:  # a component has requested a forced-end of Routine
            break
        continueRoutine = False  # will revert to True if at least one component still running
        for thisComponent in trialComponents:
            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                continueRoutine = True
                break  # at least one component has not yet finished
        
        # refresh the screen
        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen
            win.flip()
    
    # -------Ending Routine "trial"-------
    for thisComponent in trialComponents:
        if hasattr(thisComponent, "setAutoDraw"):
            thisComponent.setAutoDraw(False)
    Block1.addData('ISI.started', ISI.tStart)
    Block1.addData('ISI.stopped', ISI.tStop)
    Block1.addData('BiasImageCue.started', BiasImageCue.tStartRefresh)
    Block1.addData('BiasImageCue.stopped', BiasImageCue.tStopRefresh)
    Block1.addData('GridStimulus.started', GridStimulus.tStartRefresh)
    Block1.addData('GridStimulus.stopped', GridStimulus.tStopRefresh)
    # check responses
    if Resp.keys in ['', [], None]:  # No response was made
        Resp.keys = None
        # was no response the correct answer?!
        if str(CorrAns).lower() == 'none':
           Resp.corr = 1;  # correct non-response
        else:
           Resp.corr = 0;  # failed to respond (incorrectly)
    # store data for Block1 (TrialHandler)
    Block1.addData('Resp.keys',Resp.keys)
    Block1.addData('Resp.corr', Resp.corr)
    if Resp.keys != None:  # we had a response
        Block1.addData('Resp.rt', Resp.rt)
    Block1.addData('Resp.started', Resp.tStartRefresh)
    Block1.addData('Resp.stopped', Resp.tStopRefresh)
    if p_port.status == STARTED:
        win.callOnFlip(p_port.setData, int(0))
    Block1.addData('p_port.started', p_port.tStartRefresh)
    Block1.addData('p_port.stopped', p_port.tStopRefresh)

Coding isn’t my strength at the moment but I wonder if the ISI script is not placing the movie component correctly to be set during the ISI? Or is the only difference between the script - GridStimulus.setMovie(StimulusMovie) - in the ISI updates section of the first script the correct way to set it during the ISI period?

  1. ISI being loaded later

So, with this, I made sure to select the ISI period and use the ‘move it to the top’ option, which I believe makes it execute first? I’m not completely sure but this may be reflected in the order of components in the trialComponents variable, which does list ISI first. Interestingly, and again I don’t know a lot about the coding side of things, in the section of the script where each component updates, ISI is placed not first, but last.