Saving trial-level audio output in real time

Builder
1

Mac 12.6.2 OS (Monterrey)
PsychoPy version: 2022.2.4
Standard version

Hi all,

We’re having issues successfully running experiments in which verbal responses are recorded via microphone. Here’s a bit of background…

We present visual (e.g., a picture, a word) or auditory (e.g., a sentence or a piano melody) stimuli to participants for them to name into a microphone. We use a webcam connected via USB-C to a mac mini to audio record trials.

We have a separate microphone connected to a device that converts an utterance by the participant into a key press. That key press (equivalent to hitting the 0 key) is used as a trigger to record response time on each trial. We don’t ‘force end of routine’ when the button is pressed because that would prevent PsychoPy from recording the utterance via the webcam microphone.

Attached please find a 5-trial picture naming experiment that gives you an overview of how we collect verbal responses.

Garcea_PicNaming_Share.zip (467.6 KB)

We’re having persistent issues recording audio and saving microphone responses. Our experiments will crash randomly and audio responses will not get saved out. Others have had this issue and have documented the problem here, for example:

It appears that a short-term fix is to open PsychoPy via the terminal. Others have noticed their experiments don’t crash when using this ‘version’ of PsychoPy. Unfortunately this option does not fix our issue. We still find that experiments will crash randomly. It would appear this is specific to microphone recording because we run button-press tasks and get good data consistently.

Is there a way, at minimum, to save trial-level data in real time? I’m wondering if we could at least get the data generated prior to a crash. It’s better than nothing at all. Any pointers/tips would be much appreciated.

Thanks in advance,
FG

2

@bwinston – CC’ing you in here in case you ever figured out a solution to the issue you documented. Thanks!

3

that’s weird. when i installed the new updated psychopy a few weeks ago the microphone issue wasn’t a problem anymore (psychopy asked for microphone permissions successfully). i never experienced crashing fortunately. maybe try updating and if that doesnt work a different audio backend? there also might be a way to use the libraries they use (e.g. pyaudio) thru code components yourself. sorry if this isn’t helpful.

4

Hi Brian,

Thanks for the heads up. I wasn’t aware of version 2022.2.5. We’ll update to see if this addresses the issue.

We may need to including some code (as you suggest) to change the library. Do you know of a good place to look for code that would do the job?

Thanks all,
FG

5

Hey Brian,

Thanks again for your input. I think I figured out what our problem was. I’m documenting everything here for others to see.

Within the experiment settings GUI I needed to manually configure the sound library to PTB and I had to adjust the latency control to the -2- setting. I find that I can get verbal recordings when it is set to -3- or -4- but my experiment does occasionally crash. I made sure to adjust PsychoPy preference (cmd + ,) so that this is default moving forward.

Separately, I had to modify my microphone recording frequency from 48k to 16k. I think that my mac mini was not able to record and convert audio recordings unless I changed the default parameters to a lower frequency.

This would explain why my experiment would crash at random times and not at a specific location in the experiment. We’ll need to do more testing to figure out if this ‘solves’ the problem. e.g., others have said to run PsychoPy through the terminal. This didn’t fix my issue.

Thanks,
FG

6

Interesting! I always get poor recordings unless I have it on the highest latency, and I put the frequency on 16k (or whatever the voice setting is).

Running PsychoPy through the terminal was a workaround for a different issue—PsychoPy not asking for mic permissions. Thank you, devs, for fixing that issue!

image001.png

7

Hi everyone,

I wanted to follow up here, as we’ve made progress on this issue but have not found a long-term solution. The problem was that our picture naming task (attached above) would randomly crash.

It’s likely that we were recording at too high a temporal resolution. When we dropped it to 16k it appeared we found a solution. Problem solved! Not quite…

Now we experience a similar surface level behavior: We’ll be running the picture naming task and the experiment will just freeze in the middle of a trial. The error message states that “the Experiment ended with exit code 1”.

The problem is that this error happens randomly within an experiment, so we can’t pinpoint where the problem is coming from. The error also happens randomly between experiments. Some tasks work just fine whereas others (like the attached picture naming experiment) give us persistent difficulty.

This only happens when we run experiments with a naming response. All of our button press tasks work without issue. This suggests to me that it has something to do with the microphone component.

We are collecting incomplete datasets because of this problem, so I’m hoping to find a solution ASAP. Any input or suggestions would be extremely helpful.

8

I just encountered a different error with the same surface level behavior. In sum: A picture is presented. We record a verbal response and the trial ends when the experimenter hits a button to advance to the next trial. Trials will randomly “freeze” with the stimulus on the screen. I realized info from the log file might be helpful…

Generating PsychoPy script…

2023-03-02 12:37:28.707 python[2176:32599] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/qf/p7986zss0vgg4hqypnx504rc0000gq/T/org.opensciencetools.psychopy.savedState
Traceback (most recent call last):
File “runpy.pyc”, line 194, in _run_module_as_main
File “runpy.pyc”, line 87, in _run_code
File “/Applications/PsychoPy.app/Contents/Resources/lib/python3.8/psychopy/scripts/psyexpCompile.py”, line 253, in
compileScript(args.infile, args.version, args.outfile)
File “/Applications/PsychoPy.app/Contents/Resources/lib/python3.8/psychopy/scripts/psyexpCompile.py”, line 242, in compileScript
thisExp = _getExperiment(infile, version)
File “/Applications/PsychoPy.app/Contents/Resources/lib/python3.8/psychopy/scripts/psyexpCompile.py”, line 132, in _getExperiment
from psychopy.app.builder import experiment
File “/Users/cabin/.psychopy3/versions/psychopy/app/builder/init.py”, line 4, in
from … import experiment
File “/Users/cabin/.psychopy3/versions/psychopy/experiment/init.py”, line 20, in
from .components import getInitVals, getComponents, getAllComponents
File “/Users/cabin/.psychopy3/versions/psychopy/experiment/components/init.py”, line 18, in
from ._base import BaseVisualComponent, BaseComponent
File “/Users/cabin/.psychopy3/versions/psychopy/experiment/components/_base.py”, line 19, in
from psychopy.alerts import alerttools
File “/Users/cabin/.psychopy3/versions/psychopy/alerts/alerttools.py”, line 9, in
from psychopy.visual.textbox2.fontmanager import FontManager
File “/Users/cabin/.psychopy3/versions/psychopy/visual/init.py”, line 29, in
from psychopy.visual.backends import gamma
File “/Users/cabin/.psychopy3/versions/psychopy/visual/backends/init.py”, line 11, in
import psychopy.plugins as plugins
File “/Users/cabin/.psychopy3/versions/psychopy/plugins/init.py”, line 29, in
import psychopy.tools.pkgtools as pkgtools
File “/Users/cabin/.psychopy3/versions/psychopy/tools/pkgtools.py”, line 39, in
if prefs.paths[‘packages’] not in pkg_resources.working_set.entries:
KeyError: ‘packages’

Running: /Users/cabin/Desktop/psychopy/session001/PICTURE_NAMING_A/PICTURE_NAMING_A_lastrun.py

1867.9266 INFO Loaded monitor calibration from [‘2022_08_17 13:27’]
2023-03-02 12:37:31.703 python[2186:32680] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/qf/p7986zss0vgg4hqypnx504rc0000gq/T/org.opensciencetools.psychopy.savedState
################ Experiment ended with exit code -15 [pid:2186] ################

9

Update and Workaround

On each trial of my picture naming task the experimenter hits a button to end the trial when the participant names the picture. I moved my task from a Mac to a Dell PC and had no issue running the experiment while capturing button presses and a microphone recording of each trial.

I’m not sure what it will take to address the issue on a Mac. If I disabled the microphone I could present the pictures without issue and capture button presses; and if I disabled the keyboard I could collect microphone responses without issue. It is when both were turned on that I encountered random freezing (and sometimes no freezing at all).

Please don’t hesitate to reach out if you come across this issue in the future. I want to make sure future users can address the issue if/when they experience a similar problem.

FG

10

Hello,

I am experiencing a similar issue and working with a non-mac device is not an option at the moment:( I was wondering if you were able to figure out a way to save mic recordings every trial. Because when it gives an error or freezes we lose the data from the previous trials as it is not saving any mic responses until the experiment is completed.

Many thanks in advance!

11

The new release (coming soon) saves recordings when escape is pressed.