Creating a live webcam feed using OpenCV

Hello! I am trying to create a routine in which a live webcam is displayed on the Psychopy window. I have successfully done this standalone using OpenCV, but I am trying to fit it inside of a Psychopy window this time.

The code I use:

import cv2

webcam = cv2.VideoCapture(0)


while (True):
      
    ret, frame = webcam.read()
    cv2.imshow('Input', frame)

    c = cv2.waitKey(1)
    if c == 27:
        break

webcam.release()
cv2.destroyAllWindows()

I went into Builder and put that into a code block, and it runs normally with no problems. The issue is getting the video feed to display inside of the Psychopy window, not a separate OpenCV window. How should I go about this?

Thank you!

It’s been a while since I’ve used OpenCV so not sure about the details, but I would try the following:

  • The frame variable seems to contain the actual image. Using the standalone OpenCV version, can you check if you can display this image using matplotlib (e.g., plt.imshow(frame))? This would be a way to discover how the frame is represented
  • In Psychopy, if you have an ImageStim available, you can set a new image using video_image.SetImage(frame)
  • You might have to play around with the format in which the frame image is represented. I remember at some point having to convert pixel values which were originally presented as 8bit integers (i.e., 0 → 255) to a floating point range [-1, 1].

Hope this helps!

Thank you!

plt.imshow(frame)
plt.show()

Using this to display the webcam does work, although it only shows one frame at a time and doesn’t update live.

Bringing it into Psychopy, I created a code component and a blank image component named webcam_image, set to update each frame

In the code component I have (with the necessary imports and declarations taken care of in before experiment)

# each frame
while (True):
      
    ret, frame = webcam.read()
    webcam_image.SetImage(frame)

webcam.release()

However the program crashes without showing anything. Any ideas? thank you

Can you check in what format the frame variable is represented (e.g., numpy array, range of values)? I also think you need to initialize the blank image component with the width and the height in which the frame variable contains the image.

I thought of a weird solution to this, and it actually worked!

# each frame
ret, frame = webcam.read()
cv2.imwrite('savedframe.png', frame)
webcam_image.setImage('savedframe.png')

I save the frame image as its own png and then load in that png to the blank image component rather than the frame variable. The webcam displays perfectly inside the Psychopy window with no lag or delay so I’m very happy with this result.