Letting participants press the space bar to end microphone recording after they finish speaking, and then have the experiment advance to the next trial

I have a picture description experiment I’m running in lab using psychopy (2022.2.3), where I want to allow participants to end the recording when they are finished speaking (variable lengths) with one keypress (space). The general flow of this experiment is that they are given a verb that they have to incorporate into their description (displayed on a timer of 2 seconds, no keypress), then the picture is displayed and they can press the space bar when they’re done speaking, and then it continues to a fixation cross (displayed for 1.5 seconds, no keypress) and then the routine repeats. I have a general time limit on the microphone component of 15 seconds so that they don’t sit there and plan out their entire description before speaking, as that would go against the point of what this specific task is studying.

I have code for this mostly working, with a space button press that stops the microphone but does not force the end of the routine, and then I added some code I found to look for the space bar press and then stop the microphone and change continueRoutine to false. This has worked very well in my own testing of the experiment, but when I ran my first participant in the lab, I realized that if a participant ever runs out of time for a description or presses the space bar during the fixation cross - which is part of the trial loop but a separate routine - then things start to get wonky with the experiment and it will skip the next picture and the timers for the word that gets displayed before the picture and the fixation cross after start to get messed up and they display for a longer time, and then the space bar press for future pictures doesn’t work automatically like it did before whatever trial this started happening on. And most detrimental of all, the audio files for all descriptions recorded after whatever image this started on are the same recording as that image, leading to lots of data loss depending on if/where it happened for a given participant.

I’ve tried a bunch of different things like moving around the order of the components, lengthening the time the microphone records for to try and just prevent this from happening, lengthening the max recording size, and things in the instructions too like telling the participants to press the space bar as soon as they’ve finished speaking, but since people are humans and forget or sometimes take longer to retrieve and say a word, this happens at least once in about half my participants, and I haven’t been able to find a workable solution (other than thinking of just putting the microphone on a hard timer and not giving participants the ability to press the space bar, although I would like to have that). I think the general problem is that I’m not sure exactly what is going on in the psychopy code flow that is leading to this happening, especially all the audio files getting the same recording from one trial, so I’m not sure what to change and how to fix it.

I’ve also been trying to play around with turning the defaultKeyboard on/off using .start() and .stop(), but either I’m not putting them in the right place or it’s not working as intended because a keypress of space during the verb or fixation cross will automatically skip the picture.

The code for the routine is quite long, but I have copied it here, and I’ve include a py file of the entire experiment in case relevant information is outside the trial loop:

# set up handler to look after randomisation of conditions etc
trials = data.TrialHandler(nReps=1.0, method='sequential', 
    extraInfo=expInfo, originPath=-1,
    trialList=data.importConditions(LISTNAME, selection=pseudo_2),
    seed=None, name='trials')
thisExp.addLoop(trials)  # add the loop to the experiment
thisTrial = trials.trialList[0]  # so we can initialise stimuli with some values
# abbreviate parameter names if possible (e.g. rgb = thisTrial.rgb)
if thisTrial != None:
    for paramName in thisTrial:
        exec('{} = thisTrial[paramName]'.format(paramName))

for thisTrial in trials:
    currentLoop = trials
    # abbreviate parameter names if possible (e.g. rgb = thisTrial.rgb)
    if thisTrial != None:
        for paramName in thisTrial:
            exec('{} = thisTrial[paramName]'.format(paramName))
    
    # --- Prepare to start Routine "give_verb" ---
    continueRoutine = True
    routineForceEnded = False
    # update component parameters for each repeat
    verb_text.setText(Verb)
    # keep track of which components have finished
    give_verbComponents = [verb_text]
    for thisComponent in give_verbComponents:
        thisComponent.tStart = None
        thisComponent.tStop = None
        thisComponent.tStartRefresh = None
        thisComponent.tStopRefresh = None
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
    # reset timers
    t = 0
    _timeToFirstFrame = win.getFutureFlipTime(clock="now")
    frameN = -1
    
    # --- Run Routine "give_verb" ---
    while continueRoutine and routineTimer.getTime() < 2.0:
        # get current time
        t = routineTimer.getTime()
        tThisFlip = win.getFutureFlipTime(clock=routineTimer)
        tThisFlipGlobal = win.getFutureFlipTime(clock=None)
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        
        # *verb_text* updates
        if verb_text.status == NOT_STARTED and tThisFlip >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            verb_text.frameNStart = frameN  # exact frame index
            verb_text.tStart = t  # local t and not account for scr refresh
            verb_text.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(verb_text, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.timestampOnFlip(win, 'verb_text.started')
            verb_text.setAutoDraw(True)
        if verb_text.status == STARTED:
            # is it time to stop? (based on global clock, using actual start)
            if tThisFlipGlobal > verb_text.tStartRefresh + 2.0-frameTolerance:
                # keep track of stop time/frame for later
                verb_text.tStop = t  # not accounting for scr refresh
                verb_text.frameNStop = frameN  # exact frame index
                # add timestamp to datafile
                thisExp.timestampOnFlip(win, 'verb_text.stopped')
                verb_text.setAutoDraw(False)
        
        # check for quit (typically the Esc key)
        if endExpNow or defaultKeyboard.getKeys(keyList=["escape"]):
            core.quit()
        
        # check if all components have finished
        if not continueRoutine:  # a component has requested a forced-end of Routine
            routineForceEnded = True
            break
        continueRoutine = False  # will revert to True if at least one component still running
        for thisComponent in give_verbComponents:
            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                continueRoutine = True
                break  # at least one component has not yet finished
        
        # refresh the screen
        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen
            win.flip()
    
    # --- Ending Routine "give_verb" ---
    for thisComponent in give_verbComponents:
        if hasattr(thisComponent, "setAutoDraw"):
            thisComponent.setAutoDraw(False)
    # using non-slip timing so subtract the expected duration of this Routine (unless ended on request)
    if routineForceEnded:
        routineTimer.reset()
    else:
        routineTimer.addTime(-2.000000)
    
    # --- Prepare to start Routine "trial" ---
    continueRoutine = True
    routineForceEnded = False
    # update component parameters for each repeat
    # Run 'Begin Routine' code from code
    trial_ori = ori.pop() #takes the orientation, either LR or RL
        
    thisExp.addData('ORIENTATION' , trial_ori)
        
    if trial_ori == "RL":
        IMG = IMG_FILE_RL
    if trial_ori == "LR":
        IMG = IMG_FILE_LR
        
    nme = str(IMG)[:-4]+".wav" #to name the audio file based off the image name
    print(nme)
    
    stimulus.setImage(IMG)
    key_resp_end_mic.keys = []
    key_resp_end_mic.rt = []
    _key_resp_end_mic_allKeys = []
    # keep track of which components have finished
    trialComponents = [stimulus, key_resp_end_mic, mic]
    for thisComponent in trialComponents:
        thisComponent.tStart = None
        thisComponent.tStop = None
        thisComponent.tStartRefresh = None
        thisComponent.tStopRefresh = None
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
    # reset timers
    t = 0
    _timeToFirstFrame = win.getFutureFlipTime(clock="now")
    frameN = -1
    
    # --- Run Routine "trial" ---
    while continueRoutine and routineTimer.getTime() < 15.5:
        # get current time
        t = routineTimer.getTime()
        tThisFlip = win.getFutureFlipTime(clock=routineTimer)
        tThisFlipGlobal = win.getFutureFlipTime(clock=None)
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        
        # *stimulus* updates
        if stimulus.status == NOT_STARTED and tThisFlip >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            stimulus.frameNStart = frameN  # exact frame index
            stimulus.tStart = t  # local t and not account for scr refresh
            stimulus.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(stimulus, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.timestampOnFlip(win, 'stimulus.started')
            stimulus.setAutoDraw(True)
        if stimulus.status == STARTED:
            # is it time to stop? (based on global clock, using actual start)
            if tThisFlipGlobal > stimulus.tStartRefresh + 15.5-frameTolerance:
                # keep track of stop time/frame for later
                stimulus.tStop = t  # not accounting for scr refresh
                stimulus.frameNStop = frameN  # exact frame index
                # add timestamp to datafile
                thisExp.timestampOnFlip(win, 'stimulus.stopped')
                stimulus.setAutoDraw(False)
        
        # *key_resp_end_mic* updates
        waitOnFlip = False
        if key_resp_end_mic.status == NOT_STARTED and tThisFlip >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            key_resp_end_mic.frameNStart = frameN  # exact frame index
            key_resp_end_mic.tStart = t  # local t and not account for scr refresh
            key_resp_end_mic.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(key_resp_end_mic, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.timestampOnFlip(win, 'key_resp_end_mic.started')
            key_resp_end_mic.status = STARTED
            # keyboard checking is just starting
            waitOnFlip = True
            win.callOnFlip(key_resp_end_mic.clock.reset)  # t=0 on next screen flip
            win.callOnFlip(key_resp_end_mic.clearEvents, eventType='keyboard')  # clear events on next screen flip
        if key_resp_end_mic.status == STARTED:
            # is it time to stop? (based on global clock, using actual start)
            if tThisFlipGlobal > key_resp_end_mic.tStartRefresh + 15.5-frameTolerance:
                # keep track of stop time/frame for later
                key_resp_end_mic.tStop = t  # not accounting for scr refresh
                key_resp_end_mic.frameNStop = frameN  # exact frame index
                # add timestamp to datafile
                thisExp.timestampOnFlip(win, 'key_resp_end_mic.stopped')
                key_resp_end_mic.status = FINISHED
        if key_resp_end_mic.status == STARTED and not waitOnFlip:
            theseKeys = key_resp_end_mic.getKeys(keyList=['space'], waitRelease=False)
            _key_resp_end_mic_allKeys.extend(theseKeys)
            if len(_key_resp_end_mic_allKeys):
                key_resp_end_mic.keys = _key_resp_end_mic_allKeys[-1].name  # just the last key pressed
                key_resp_end_mic.rt = _key_resp_end_mic_allKeys[-1].rt
        
        # mic updates
        if mic.status == NOT_STARTED and t >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            mic.frameNStart = frameN  # exact frame index
            mic.tStart = t  # local t and not account for scr refresh
            mic.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(mic, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.addData('mic.started', t)
            # start recording with mic
            mic.start()
            mic.status = STARTED
        if mic.status == STARTED:
            # update recorded clip for mic
            mic.poll()    
        if mic.status == STARTED:
            if event.getKeys(['space']):
                mic.stop()
                mic.status == FINISHED
                continueRoutine = False
        if mic.status == STARTED:
            # is it time to stop? (based on global clock, using actual start)
            if tThisFlipGlobal > mic.tStartRefresh + 15.0-frameTolerance:
                # keep track of stop time/frame for later
                mic.tStop = t  # not accounting for scr refresh
                mic.frameNStop = frameN  # exact frame index
                # add timestamp to datafile
                thisExp.addData('mic.stopped', t)
                # stop recording with mic
                mic.stop()
                mic.status = FINISHED
        
        # check for quit (typically the Esc key)
        if endExpNow or defaultKeyboard.getKeys(keyList=["escape"]):
            core.quit()
        
        # check if all components have finished
        if not continueRoutine:  # a component has requested a forced-end of Routine
            routineForceEnded = True
            break
        continueRoutine = False  # will revert to True if at least one component still running
        for thisComponent in trialComponents:
            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                continueRoutine = True
                break  # at least one component has not yet finished
        
        # refresh the screen
        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen
            win.flip()
    
    # --- Ending Routine "trial" ---
    for thisComponent in trialComponents:
        if hasattr(thisComponent, "setAutoDraw"):
            thisComponent.setAutoDraw(False)
    # check responses
    if key_resp_end_mic.keys in ['', [], None]:  # No response was made
        key_resp_end_mic.keys = None
    trials.addData('key_resp_end_mic.keys',key_resp_end_mic.keys)
    if key_resp_end_mic.keys != None:  # we had a response
        trials.addData('key_resp_end_mic.rt', key_resp_end_mic.rt)
    # tell mic to keep hold of current recording in mic.clips and transcript (if applicable) in mic.scripts
    # this will also update mic.lastClip and mic.lastScript

    tag = data.utils.getDateStr()
    
    audioClip = mic.getRecording()
    print(audioClip.duration)
    print(os.path.join(micRecFolder, '%s_%s.wav' % (TRIALCOUNTER,nme)))
    audioClip.save(os.path.join(micRecFolder, '%s_%s.wav' % (TRIALCOUNTER, nme)))
    TRIALCOUNTER += 1
    trials.addData('mic.clip', os.path.join(micRecFolder, 'recording_mic_%s.wav' % tag))
    # the Routine "trial" was not non-slip safe, so reset the non-slip timer
    if routineForceEnded:
        routineTimer.reset()
    else:
        routineTimer.addTime(-12.500000)
    
    # --- Prepare to start Routine "break_2" ---
    continueRoutine = True
    routineForceEnded = False
    # update component parameters for each repeat
    key_resp8.keys = []
    key_resp8.rt = []
    _key_resp8_allKeys = []
    # keep track of which components have finished
    break_2Components = [key_resp8, polygon]
    for thisComponent in break_2Components:
        thisComponent.tStart = None
        thisComponent.tStop = None
        thisComponent.tStartRefresh = None
        thisComponent.tStopRefresh = None
        if hasattr(thisComponent, 'status'):
            thisComponent.status = NOT_STARTED
    # reset timers
    t = 0
    _timeToFirstFrame = win.getFutureFlipTime(clock="now")
    frameN = -1
    
    # --- Run Routine "break_2" ---
    while continueRoutine and routineTimer.getTime() < 1.5:
        # get current time
        t = routineTimer.getTime()
        tThisFlip = win.getFutureFlipTime(clock=routineTimer)
        tThisFlipGlobal = win.getFutureFlipTime(clock=None)
        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
        # update/draw components on each frame
        
        # *key_resp8* updates
        waitOnFlip = False
        if key_resp8.status == NOT_STARTED and tThisFlip >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            key_resp8.frameNStart = frameN  # exact frame index
            key_resp8.tStart = t  # local t and not account for scr refresh
            key_resp8.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(key_resp8, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.timestampOnFlip(win, 'key_resp8.started')
            key_resp8.status = STARTED
            # keyboard checking is just starting
            waitOnFlip = True
            win.callOnFlip(key_resp8.clock.reset)  # t=0 on next screen flip
            win.callOnFlip(key_resp8.clearEvents, eventType='keyboard')  # clear events on next screen flip
        if key_resp8.status == STARTED and not waitOnFlip:
            theseKeys = key_resp8.getKeys(keyList=['p'], waitRelease=False)
            _key_resp8_allKeys.extend(theseKeys)
            if len(_key_resp8_allKeys):
                key_resp8.keys = _key_resp8_allKeys[-1].name  # just the last key pressed
                key_resp8.rt = _key_resp8_allKeys[-1].rt
                # a response ends the routine
                #continueRoutine = False
        if event.getKeys(['p']):
            time.sleep(20)
        # *polygon* updates
        if polygon.status == NOT_STARTED and tThisFlip >= 0.0-frameTolerance:
            # keep track of start time/frame for later
            polygon.frameNStart = frameN  # exact frame index
            polygon.tStart = t  # local t and not account for scr refresh
            polygon.tStartRefresh = tThisFlipGlobal  # on global time
            win.timeOnFlip(polygon, 'tStartRefresh')  # time at next scr refresh
            # add timestamp to datafile
            thisExp.timestampOnFlip(win, 'polygon.started')
            polygon.setAutoDraw(True)
        if polygon.status == STARTED:
            # is it time to stop? (based on global clock, using actual start)
            if tThisFlipGlobal > polygon.tStartRefresh + 1.5-frameTolerance:
                # keep track of stop time/frame for later
                polygon.tStop = t  # not accounting for scr refresh
                polygon.frameNStop = frameN  # exact frame index
                # add timestamp to datafile
                thisExp.timestampOnFlip(win, 'polygon.stopped')
                polygon.setAutoDraw(False)
        # check for quit (typically the Esc key)
        if endExpNow or defaultKeyboard.getKeys(keyList=["escape"]):
            core.quit()
        
        # check if all components have finished
        if not continueRoutine:  # a component has requested a forced-end of Routine
            routineForceEnded = True
            break
        continueRoutine = False  # will revert to True if at least one component still running
        for thisComponent in break_2Components:
            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:
                continueRoutine = True
                break  # at least one component has not yet finished
        
        # refresh the screen
        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen
            win.flip()
    
    # --- Ending Routine "break_2" ---
    for thisComponent in break_2Components:
        if hasattr(thisComponent, "setAutoDraw"):
            thisComponent.setAutoDraw(False)
    # check responses
    if key_resp8.keys in ['', [], None]:  # No response was made
        key_resp8.keys = None
    trials.addData('key_resp8.keys',key_resp8.keys)
    if key_resp8.keys != None:  # we had a response
        trials.addData('key_resp8.rt', key_resp8.rt)
    # the Routine "break_2" was not non-slip safe, so reset the non-slip timer
    routineTimer.reset()
    thisExp.nextEntry()
    
# completed 1.0 repeats of 'trials'

I know I could also do something like set a mouse click instead or have a button box that I control, but I would prefer to find a solution that will work regardless of what the response is just in case, and that will allow the participant to end their own recording when they decide they have finished speaking.

Basically if possible it’d be great to disable/not count any responses made on either the fixation cross page or the verb page, I’m just not sure how to do that specifically.

dative_experiment.py (59.0 KB)
. Thank you!