Using Sound and Microphone components in the same experiment crashes experiment (ubuntu)

OS (e.g. Win10): Ubuntu 22.04.4 LTS
PsychoPy version : 2024.2.1 and 2024.2.0 (have not tested older versions yet)
Standard Standalone? (y/n) na
What are you trying to achieve?: Trying to play audio files for participants who will then repeat them back (phonetic shadowing task)

What did you try to make it work?: I have tried installing in a python3.10 venv (following current instructions on psychopy.org and using the installer from github) and building a miniconda environment using python 3.8.19

What specifically went wrong when you tried that?:

A grad student in our lab is trying to run a shadowing task using psychopy. He’s created a stripped-down experiment with 2 routines: the first uses a Sound component to play a 400 Hz tone (the real experiment uses wav files, but the demonstration breaks even with just a tone). The second routine uses a Microphone component to record the participant’s response (ideally this should be mono, 44.1 kHz or 48 kHz). If both components are active, the experiment crashes at launch. If either component is disabled for testing, the experiment runs and either plays sound or records sound as expected. The Microphone demo with whisper transcription enabled works on these machines. We have tried this with two usb audio interfaces: a Steinberg UR-22 and a MOTU M6 and both crash. We’re using the ptb audio library. Results are the same regardless of latency level in psychopy settings. He has also tested this experiment on Windows and it works there, but we don’t have Windows on the lab machines.

Here is what I believe to be the relevant part of the error message when both components are enabled:

  self.handle = PsychPortAudio('Open', device_id, mode,
Exception: Failed to open PortAudio audio device due to unsupported combination of audio parameters.
Exception ignored in: <function Stream.__del__ at 0x7fcb4841b5e0>
PTB-ERROR: Seems the requested audio sample rate 48000.000000 Hz is not supported by this combo of hardware and sound driver.
PTB-ERROR: On Linux you may be able to use ALSA audio converter plugins to make this work.
Error in function Open:     Usage error
Failed to open PortAudio audio device due to unsupported combination of audio parameters.
PsychPortAudio:Open: Usage:

pahandle = PsychPortAudio('Open' [, deviceid][, mode][, reqlatencyclass][, freq][, channels][, buffersize][, suggestedLatency][, selectchannels][, specialFlags=0]);
PTB-ERROR: Desired audio parameters for device -1 unsupported by audio device: Invalid sample rate 
PTB-ERROR: Seems the requested audio sample rate 48000.000000 Hz is not supported by this combo of hardware and sound driver.
PTB-ERROR: On Linux you may be able to use ALSA audio converter plugins to make this work.
Error in function Open:     Usage error
Failed to open PortAudio audio device due to unsupported combination of audio parameters.
PsychPortAudio:Open: Usage:

pahandle = PsychPortAudio('Open' [, deviceid][, mode][, reqlatencyclass][, freq][, channels][, buffersize][, suggestedLatency][, selectchannels][, specialFlags=0]);
5.8595     WARNING     Could not find exact match for specified parameters (index=0, sampleRateHz=44100, channels=None), falling back to best approximation (index=0, sampleRateHz=44100, channels=2)

The Microphone component is configured to capture at 44.1 kHz mono (changing to auto or stereo results in the same error). Changing it to 48 kHz (on the suspicion that the Sound component is opening a portaudio connection at 48 kHz) still crashes. I can’t find anywhere at all in this experiment or in the psychopy preferences or Ubuntu settings where 48 kHz is being requested or configured, but I can verify that on this machine, with these hardware interfaces, the program Audacity can record 44.1 kHz and 48 kHz either mono or stereo (signal only on the left channel because it’s a mono mic) when configured to the same portaudio device we’re specifying in PsychoPy preferences.

Does anyone have any suggestions for getting sound playback and recording (non-simultaneous) working in a single Psychopy experiment running on Ubuntu? Perhaps the problem is that we’re using the ptb driver? Any advice greatly appreciated!

I’m attaching the demonstration experiment that crashes on our Ubuntu 22.04 lab machines.
sound_mic.psyexp (18.9 KB)

That’s strange indeed…
I just tried out your experiment file on a Debian 12 system with PsychoPy 2024.2.1 in a Python 3.8 virtual environment, using the ptb audio library, and it worked fine – both playing the beep and recording my response at 44.1 kHz.

After a few more tries:

I sometimes see the experiment terminate after asking for the participant ID, though, depending on various audio (PulseAudio) settings. The only error messages I could elicit are of this type:

  File ".../psypy/lib/python3.8/site-packages/psychopy/hardware/manager.py", line 253, in addDevice
    raise ManagedDeviceError(
psychopy.hardware.manager.ManagedDeviceError: Could not find any device with index 0
Failure: No such entity
1 Like

thank you so much for trying it on Debian! Can I ask what your audio hardware is like?

It’s this:

AMD Starship/Matisse HD Audio driver: snd_hda_intel
1 Like

P.S.: Have you tried with both 44.1 kHz and 48 kHz sampling settings?
This caused me some hours of frustration; sound played ok when running the experiment in the browser but not in PsychoPy.
Your error messages seem to hint at a sample rate issue. – But I’m only guessing, ofc…

Just out of curiosity, how do you set the sample rate? I’m not convinced I know how.

I have tried with different sampling rates, but I don’t know of a way to set it on the Sound component (apart from opening a wav file of the specified rate) and I don’t feel confident that the sample rate selection on the Microphone component is doing anything.

I’ve tried playing a 41000 Hz file and a 48000 Hz file with the corresponding sample rate selected for the Microphone component, but it crashes with the same error about 48000 not being a supported rate by this device (it is, I can record 48k audio in other software).

You’re so right…; it got more complicated for me too once I started to try out different playing and recording options a little more systematically.

The interplay of psychtoolbox, PsychoPy and pipewire/PulseAudio/ALSA seems to be rather complex – but I’m afraid I won’t get back to you with more concrete info before the weekend or early next week, since I am abroad at a conference right now (teaching a workshop on, of all things, Phonetic experiments with PsychoPy and PsychoJS :rofl:).

I’ll try to come up by then with at least a detailed report on a setup with which I managed to run your experiment reliably and repeatedly.

Just quickly about Sound components’ sample rate: no, as you say, you can’t override that with a PsychoPy setting, it just takes what it finds in the sound file.

But: sometimes it will use a different rate anyway (as I mentioned in an earlier post), most likely due to some psychtoolbox peculiarities; then you’ll just have to resample to the sampling frequency it seems to want, and probably also adjust the recording (Microphone) sample rate accordingly.

This is very helpful, thank you. I’m digging around in alsa settings, asound.conf, ptb forums, I put debian 12 on a test machine, and I’ve ordered a focusrite solo because they did a bunch of work last year to incorporate drivers in the linux kernel.

This workshop sounds awesome, I hope it’s going well!

I think this might be related to just general problems with ptb. First, I found this bug report that feels relevant: BF: Fix default audio device not being found by TEParsons · Pull Request #6863 · psychopy/psychopy · GitHub

so then I tried backing out to 2023.2.3 which still crashes but gives the error:


PTB-ERROR: Failed to open audio device 0. PortAudio reports this error: Device unavailable 
Expression 'ret' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1742
PTB-ERROR: Could not open audio device, most likely because it is already in exclusive use by a previous call
PTB-ERROR: to PsychPortAudio('Open', ...). You can open each exclusive device only once per session. If you need
PTB-ERROR: multiple independent devices simulated on one physical audio device, look into use of audio
PTB-ERROR: slave devices. See help for this by typing 'PsychPortAudio OpenSlave?'.
Error in function Open:     Usage error
Audio device unavailable. Most likely tried to open device multiple times.
PsychPortAudio:Open: Usage:

So I’d like to switch to pyo, sounddevice, or some other library but, as far as I can tell, ptb is hard-coded in Builder now? I can change the library in the UI but I always get the same PTB errors? Perhaps I’m losing my mind?

Don’t know about this particular problem. I often get the ptb errors below but the experiment plays audio exactly as it is supposed to anyway.
(FWIW)

PTB-ERROR: Desired audio parameters for device 1 unsupported by audio device: Invalid device 
PTB-ERROR: This could be, e.g., due to an unsupported combination of audio sample rate, audio channel count/allocation, or audio sample format.
PTB-ERROR: On Linux you may be able to use ALSA audio converter plugins to make this work.

Thank you for the good wishes – yes, I think at least some of the workshop participants were encouraged to start their own experiments now.

Please find enclosed PsychoPy’s output from a successful run of your experiment.
As promised, I will send more info once I am back home.

data.zip (266.4 KB)

P.S.: I did not change anything in your experiment, not even the Steinberg UR22 hardware preference setting in your code component, but ptb/PsychoPy did not complain about it.

I think I might have made a tiny bit of progress…
Now at home I tried your experiment on my desktop computer which has an almost identical setup to my laptop, except for kernel version (6.7 instead of 6.9) and audio server: PipeWire instead of PulseAudio: your experiment stops running after the monitor calibration with the error message Could not find any device with index 0.
I enclose the .log etc. files again (btw, no difference between running under Python 3.8 or Python 3.10).
data_pipewire.zip (2.5 KB)

The working setup thus was:

Audio:
  [...]
  Device-2: AMD Family 17h/19h HD Audio driver: snd_hda_intel
  API: ALSA v: k6.9.7+bpo-amd64 status: kernel-api
  Server-1: PulseAudio v: 16.1 status: active

and the non-working is:

Audio:
  [...]
  Device-2: AMD Starship/Matisse HD Audio driver: snd_hda_intel
  API: ALSA v: k6.7.12+bpo-amd64 status: kernel-api
  Server-1: PipeWire v: 1.2.1 status: active

Are you running PipeWire?

Follow-up:

To confirm my suspicion I uninstalled PipeWire from my desktop computer. As expected, now your experiment runs just fine.

Audio:
  [...]
  Device-2: AMD Starship/Matisse HD Audio driver: snd_hda_intel
  API: ALSA v: k6.7.12+bpo-amd64 status: kernel-api
  Server-1: PulseAudio v: 16.1 status: active
1 Like

thank you so much for all of your efforts and insight! I’m pleased both that you’re able to get this to work and that it can also sometimes break in your environment.

I disabled PipeWire with sysctl and checked with pactl to ensure it was disabled and trying to record & playback still crashes on my lab machines. This was on Ubuntu 22.04 with a v6.8 HWE lowlatency 6.8 kernel.

Still, this gives me several things to test and I tremendously appreciate it.

I’m probably being paranoid here – but are you sure you’re not still running PipeWire’s emulation of PulseAudio? (That’s what happened to me when I first tried disabling PipeWire, and in the end I didn’t find a better working solution than uninstalling all the PipeWire-related packages. But as I said, I might just have become overly terrified by PipeWire… :face_with_peeking_eye:)

1 Like

Everything claimed pipewire wasn’t running, but this turns out not to have been true! The only OSes I had access to yesterday were Ubuntu 22.04 and Pop! OS 22.04. Uninstalling pipewire also removes the gnome desktop environment and gdb so I didn’t want to mess with that.

I just tried your solution on Debian 12 with a minimal KDE Plasma desktop using X11 and only pulseaudio installed on the machine and everything works! Things are not quite so rosy on Ubuntu 22.04 where even with pulsaudio/kde things are still crashing but, frankly, I’m kind of tired of Ubuntu anyway and I miss Debian, so…

I am so thrilled and I can’t possibly thank you enough for your friendly, helpful persistence.

1 Like