psychopy.org | Reference | Downloads | Github

Add sound to webcam capture

Hello,

I’ve got an experiment I’ve designed in Coder view that captures both audio and video of participants. It then saves a .wav and .avi file; however, I was wondering if there is a way to merge the two so that the video contains the audio in it? So far I’ve only founds discussions about collecting audio or video separately.

Here’s a snippet of my code

#Set base variables:
width, height = Aspect_Ratio_Handler(w_pix, h_pix)
text = visual.TextStim(win=win, text='', height=height/22, pos=(0,0), color="White")
image = visual.ImageStim(win=win, pos=(0,0), size=(0.55,0.55), image=stim_df['Stim_Path'][0])
microphone.switchOn()
mic = microphone.AdvAudioCapture()
width, height = [image.size[0], image.size[1]]

#Blank screen before start of trials:
cumulativeTimer.reset()

#Begin looping through trials:
for i in range(0, len(stim_df)):
    mic.reset()
    escape_option()
    trialTimer.reset()
    cumulative_onset = cumulativeTimer.getTime()
    trial_onset = trialTimer.getTime()
    audio_path = '{}/Data/sub-{}/sub-{}_pic-{}'.format(cwd, expInfo['participant ID'], expInfo['participant ID'], stim_df['Stim_Name'][i].split('/')[-1].split('.')[0])
    video_path = '{}/Data/sub-{}/sub-{}_pic-{}'.format(cwd, expInfo['participant ID'], expInfo['participant ID'], stim_df['Stim_Name'][i].split('/')[-1].split('.')[0])
    theseKeys = []
    stim_duration = 'N/A'
    
    #Fixation period (1-sec):
    fixation(1)
    
    #Begin Video Recording:
    cap = cv2.VideoCapture(0 + cv2.CAP_DSHOW)
    fourcc = cv2.VideoWriter_fourcc(*'XVID')
    out = cv2.VideoWriter('{}.avi'.format(video_path),fourcc, 20.0, (640,480))
    
    #Begin audio recording:
    mic.record(sec=30, filename=audio_path)
    
    #Present Image (max 30 seconds, or until button press):
    image.image = stim_df['Stim_Path'][i]
    stim_onset = trialTimer.getTime()
    image.draw()
    win.flip()

    while trialTimer.getTime() < 30 + 1: #Max length of 30 seconds (the extra one is from the fixation period)
        escape_option()
        ret, frame = cap.read()
        out.write(frame)
        
        theseKeys = event.getKeys(keyList=["return"])
        if len(theseKeys):
            break
    
    #Stop audio & video recording after each trial:
    cap.release()
    out.release()
    cv2.destroyAllWindows()
    stim_duration = trialTimer.getTime() - 1
    mic.stop()
    

Thanks for the assistance,

Dan

Solution seems to be to use ffmpeg