Running independent visual and auditory sitmuli simultaneously

Hi there,

I am trying to set up an experiment in which a continuous movie without audio track is played during a run that includes different trials. To be more specific, I have several 8 minute long runs in my experiment that each include 40 auditory stimuli and distinct delays. Each of these runs (and its specific stimulus order) is imported from a .csv-file. Now I want an 8 minute long movie sequence to be played continuously during each run while the different trials (= auditory stimuli) and delays are presented. Movie and trials should be two independent “streams” and only for the trials it is important that they are written to a .csv-file and .log-file (stimulus type, onset, duration, delay etc.).

Till now, I have managed to include the movie in my run-routine in the experiment and also to start the first auditory stimulus during the movie. But then Psychopy does not seem to loop over different trials, but the auditory stimulation stops after the first stimulus, the movie is played till its finished and the experiment breaks. I do not know why this happens and have no idea how my “trial-loop” during the movie could run. Without including the movie in my experiment, everything worked fine.

 # ------Prepare to start Routine "movie"-------
    t = 0
    movieClock.reset()  # clock
    frameN = -1
    continueRoutine = True
    routineTimer.add(480.000000)
    # update component parameters for each repeat
    # keep track of which components have finished
    movieComponents = [movie]
    for thisComponent in movieComponents:
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
                
    continueRoutine2 = True
    # -------Start Routine "movie"-------
    while continueRoutine and routineTimer.getTime() > 0:
        # get current time
        t = movieClock.getTime()
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        movieComponents = [movie]
        
        # *movie* updates
        if t >= 0.0 and movie.status == NOT_STARTED:
            # keep track of start time/frame for later
            movie.tStart = t
            movie.frameNStart = frameN  # exact frame index
            movie.setAutoDraw(True)
        frameRemains = 0.0 + 8.0- win.monitorFramePeriod * 0.75  # most of one frame period left
        if movie.status == STARTED and t >= frameRemains:
            movie.setAutoDraw(False)
        
        	# set up handler to look after randomisation of conditions etc
        trials = data.TrialHandler(nReps=1, method='sequential', 
            extraInfo=expInfo, originPath=-1,
            trialList=data.importConditions("run%i.csv" % run_id),
            seed=None, name='trials')
        thisExp.addLoop(trials)  # add the loop to the experiment
        thisTrial = trials.trialList[0]  # so we can initialise stimuli with some values
       	# abbreviate parameter names if possible (e.g. rgb = thisTrial.rgb)
   
        if thisTrial != None:
            for paramName in thisTrial.keys():
                exec(paramName + '= thisTrial.' + paramName)

        for thisTrial in trials:
            currentLoop = trials
            # abbreviate parameter names if possible (e.g. rgb = thisTrial.rgb)
            if thisTrial != None:
                for paramName in thisTrial.keys():
                    exec(paramName + '= thisTrial.' + paramName)

            # ------Prepare to start Routine "trial"-------
            t2 = 0
            trialClock.reset()  # clock
            frame2N = -1
            continueRoutine2 = True
            # update component parameters for each repeat
            trial_start_timestamp = StartClock.getTime() - run_start_timestamp
            sample.setSound("stimuli/"+stim, secs=6.1)
            trigger_sync = event.BuilderKeyResponse()
            # keep track of which components have finished
            trialComponents = [sample, trigger_sync]
            for thisComponent in trialComponents:
                if hasattr(thisComponent, 'status'):
                    thisComponent.status = NOT_STARTED
            print("hier")
        
            # -------Start Routine "trial"-------
            while continueRoutine2:
                # get current time
                t2 = trialClock.getTime()
                frame2N = frame2N + 1  # number of completed frames (so 0 is the first frame)

                # start/stop sample
                if t >= 0.0 and sample.status == NOT_STARTED:
                    # keep track of start time/frame for later
                    sample.tStart = t
                    sample.frameNStart = frameN  # exact frame index
                    sample.setVolume(total_vol) # individual volume
                    sample.play()  # start the sound (it finishes automatically)
                frame2Remains = 0.0 + 6.1- win.monitorFramePeriod * 0.75  # most of one frame period left
                if sample.status == STARTED and t >= frameRemains:
                    sample.stop()  # stop the sound (if longer than duration)
            
                # *trigger_sync* updates
                if (t > (6 + delay -1)) and trigger_sync.status == NOT_STARTED:
                    # keep track of start time/frame for later
                    trigger_sync.tStart = t
                    trigger_sync.frameNStart = frameN  # exact frame index
                    trigger_sync.status = STARTED
                    # keyboard checking is just starting
                    win.callOnFlip(trigger_sync.clock.reset)  # t=0 on next screen flip
                    event.clearEvents(eventType='keyboard')
                if trigger_sync.status == STARTED:
                    theseKeys = event.getKeys(keyList=['t'])
                
                    # check for quit:
                    if "escape" in theseKeys:
                        endExpNow = True
                    if len(theseKeys) > 0:  # at least one key was pressed
                        trigger_sync.keys = theseKeys[-1]  # just the last key pressed
                        trigger_sync.rt = trigger_sync.clock.getTime()
                        # a response ends the routine
                    
            
                    # check for quit:
                    if "escape" in theseKeys:
                        endExpNow = True
            
                # check if all components have finished
                if not continueRoutine2:  # a component has requested a forced-end of Routine
                    break
                continueRoutine2 = False  # will revert to True if at least one component still running
                for thisComponent in trialComponents:
                    if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                        continueRoutine2 = True
                        break  # at least one component has not yet finished
            
                # check for quit (the Esc key)
                if endExpNow or event.getKeys(keyList=["escape"]):
                    core.quit()
            
                # refresh the screen
                if continueRoutine2:  # don't flip if this routine is over or we'll get a blank screen
                    win.flip()
    
            # -------Ending Routine "trial"-------
            for thisComponent in trialComponents:
                if hasattr(thisComponent, "setAutoDraw"):
                    thisComponent.setAutoDraw(False)
            trials.addData('trial_start_run_time', trial_start_timestamp)
            sample.stop()  # ensure sound has stopped at end of routine
            # check responses
            if trigger_sync.keys in ['', [], None]:  # No response was made
                trigger_sync.keys=None
            trials.addData('trigger_sync.keys',trigger_sync.keys)
            if trigger_sync.keys != None:  # we had a response
                trials.addData('trigger_sync.rt', trigger_sync.rt)
            routineTimer.reset()
            thisExp.nextEntry()
    
        # get names of stimulus parameters
        if trials.trialList in ([], [None], None):
            params = []
        else:
            params = trials.trialList[0].keys()
        # save data for this loop
        trials.saveAsText(filename + 'trials.csv', delim=',',
            stimOut=params,
            dataOut=['n','all_mean','all_std', 'all_raw'])
            
    # -------Ending Routine "movie"-------
    for thisComponent in movieComponents:
        if hasattr(thisComponent, "setAutoDraw"):
	    thisComponent.setAutoDraw(False)

Has anybody an idea how to change the code so that it will run and loop over trials?
I am using PsychoPy v1.85.3 on Ubuntu 16.04.

Thanks in advance!

Hi Mira,

The issue here is that you are using code generated by Builder, which has a very specific way of doing things. In particular, it will start and stop all stimuli within a routine: it has no concept of stimuli that span routines, or successive iterations of a routine.

What you should do is strip out all of the Builder-generated movie code and add your own that creates the movie stimulus before the routine begins, and then just issue a movie.draw() on every screen refresh.

It can be a bit clumsy figuring out how to do that in Builder-generated code, though. Is there a reason why you are editing this code rather than working within the Builder interface and just using code components to modify the way it runs? I can suggest how to implement what you need from within the Builder GUI but am not so keen to get my hands dirty with this raw code, as its verbosity makes it easy to misplace things.

Hi Michael,

thank you for reminding me that I could simply try using the Builder GUI. I got the script from a colleague and was so focused on transforming the code that I didn’t really thought about this possibility as I am also new to psychopy and didn’t have much experience in it. Using the Builder, the problem turned out to be much easier than I thought and I figured out how to implement the movie correctly. However - thank you for your advice, I don’t know how long I would have been trying some complicated code transformation otherwise.

1 Like