Hello,
I’ve created an experiment that requires video recordings of participants’ pronunciations. When I run the experiment in Builder, the resulting video files look perfect—they are stored in MP4 format with sound.
However, the experiment is intended to run online, and when it does, the video is recorded in WEBM format as expected. The problem is that the recorded video has no associated sound.
Does anyone know how to resolve this issue?
Thanks!
Hello everyone who might know the answer—and apologies for specifically tagging you, @wakecarter and @JensBoelte, but you’ve both kindly helped me resolve a few issues in the past.
Apologies for bringing this up again, but I’ve been struggling with the Camera Response component. I’ve noticed that in the latest PsychoPy version, it’s no longer marked as “Beta”. However, when I attempt to gather a camera response (i.e., participants video-recording themselves pronouncing words, as we’re interested in their facial expressions) via Pavlovia, all the resulting video files in WEBM contain only the video—there’s no audio. When I run the experiment locally in Builder, the video recordings in MP4 come out just fine.
I understand that it’s possible to add a microphone component in the same Routine to record audio separately, but this isn’t ideal for our purposes.
Could anyone clarify whether the Camera component is currently expected to record video without sound? Or is there a way to ensure that the video files include audio as well?
Many thanks in advance for your help!
From what I can tell looking at the code, the online version of the camera component is only video, but you should be able to create a separate microphone component to record audio, and as long as they have the same start time they should at least be synchronized.
The reason it works when you run it locally is that the Python version of the camera component works very differently, and it can associate a microhpone with the camera so it can record video and audio together, but the online version can’t do that at the moment.
Hi @jonathan.kominsky,
Thank you so much for your response—I really appreciate it! It’s incredibly helpful to have that clarification, as I was starting to worry I might’ve done something wrong on my end.
I know it might be a bit of an ask, but would you happen to know of any code or method that could help combine a camera WEBM file and a microphone WEBM file into a single file for each trial, when participants’ responses are logged on Pavlovia?
I’m guessing it wouldn’t be feasible to store those on the Pavlovia server due to the large file sizes, but I was wondering if there might be a way to specify an alternative web location for storing them.
Thanks again – I really appreciate your help!
The file size is probably not the issue, in the end it’s going to be similar to or slightly smaller than separate video and audio files. The issue is that the way that PsychoJS interfaces with cameras and microphones just doesn’t allow it.
It actually works almost exactly the same way in both cases, it creates a media buffer of the specified format (video or audio) and then saves it as a webm file (yes, the microphone recordings are also webm files). The problem is just that it doesn’t have a way of combining those two buffers into a single file. You can do it after downloading the data without too much trouble, you just can’t do it in Pavlovia itself as far as I can tell (even with custom code it would be very tricky because you’d have to access the buffer before it was saved and uploaded, and I’m not sure if the encoding system would allow it anyway).
I think that the PsychoJS version of the camera component is still somewhat under development (right now it’s basically a copy/paste of the microphone component with a few tweaks), so I expect this is something that may change in the future, but I’m afraid that for right now this is just a hard limitation.
@jonathan.kominsky – thank you so much for letting me know—it’s really reassuring to hear that. Yes, it would be amazing if a future version of PsychoJS could account for this. That’s because when you have to manually sync two separate files for analysis, there’s always a risk of slight misalignment, which can interfere with interpreting results—especially in studies which measure facial characteristics during language production.
I really appreciate all the helpful responses I’ve received. It’s been incredibly valuable. Thanks so much again!