Audio issue with all libraries on Windows: "doesn't match broadcast shape"

Coding
1

This has been reported before but it turns out to be much weirder than it looks: Error: non-broadcastable output during sound exp

Basically, I have a PyHab experiment (my add-on for PsychoPy). On windows, the first time I run it, it plays all the audio fine. Subsequent runs produce the following error with sounddevice, which appears as a pop-up message but does not crash the experiment (but no sound will play):

From cffi callback <function _StreamBase.__init__.<locals>.callback_ptr at 0x0000021E1D8E77B8>:
Traceback (most recent call last):
  File "C:\Program Files\PsychoPy3\lib\site-packages\sounddevice.py", line 740, in callback_ptr
    return _wrap_callback(callback, data, frames, time, status)
  File "C:\Program Files\PsychoPy3\lib\site-packages\sounddevice.py", line 2516, in _wrap_callback
    callback(*args)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 208, in callback
    toSpk[:len(dat), :] += dat  # add to out stream
ValueError: non-broadcastable output operand with shape (128,1) doesn't match the broadcast shape (128,2)

If I use PTB instead, it does crash outright, when it loads (before it attempts to play) with a similar error:

  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\contrib\lazy_import.py", line 120, in __call__
    return obj(*args, **kwargs)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py", line 135, in __init__
    self.loadMovie(self.filename)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\visual\movie3.py", line 197, in loadMovie
    sampleRate=self._mov.audio.fps)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\sound\backend_ptb.py", line 334, in __init__
    hamming=self.hamming)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\sound\backend_ptb.py", line 413, in setSound
    _SoundBase.setSound(self, value, secs, octave, hamming, log)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\sound\_base.py", line 198, in setSound
    self._setSndFromArray(numpy.array(value))
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychopy\sound\backend_ptb.py", line 486, in _setSndFromArray
    volume=self.volume)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychtoolbox\audio.py", line 300, in __init__
    self.fill_buffer(data)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychtoolbox\audio.py", line 242, in fill_buffer
    self.buffer = Buffer(stream=self, data=data)
  File "C:\Program Files\PsychoPy3\lib\site-packages\psychtoolbox\audio.py", line 260, in __init__
    data)
Exception: Number of columns of audio data matrix doesn't match number of output channels of selected audio device.

The truly perplexing thing is that it works once and then throws this error, but even deleting pycache doesn’t fix it, and deleting the app config files only fixes it once. It can’t be modifying the files themselves, so I have not the faintest clue what’s causing it to suddenly and irreparably fail.

Notably, it specifically seems to happen in cases when I load more than one movie or audio file, even though I am only attempting to play one file at a time, so it’s possible this is a memory allocation issue but that would be even weirder. Basically if I take one of the movie files that produces this error and run it in MovieStim, it works fine every time.

@jon @dvbridges or @michael, any suggestions? I’ve tried different sound libraries, different sound drivers, pretty much every audio setting I can think of, making sure the audio files are both mono and stereo, etc. Again, the file works in other contexts, but the nature of this error is utterly mystifying to me.

2

It’s something to do with a mono sound trying to play on a stereo sound stream (or vice versa). It should certainly be possible to convert the sound very easily to stereo from mono (the numpy broadcasting mechanism should just duplicate the sound to both channels I believe) so maybe this issue is the other way around that you’ve got a mono sound stream (e.g. because the first sound to play was mono) and then a stereo sound stream won’t play to it.

3

Seems to be the other way around:

ValueError: non-broadcastable output operand with shape (128,1) doesn't match the broadcast shape (128,2)

“output operand” should be the file, “broadcast shape” should be the output, so it’s an issue of mono trying to broadcast over stereo. However I get the same error when trying to play files that are mono or stereo, my output is assuredly stereo, and either way, the fact that it isn’t just duplicating the stream is a pain. Ideas?

4

No, I’m sure that automatically converting a mono stream to stereo did previously work. I guess the clue might be in the phrase “non-broadcastable output” but I don’t know what would make an array non-broadcastable. Maybe something has changed in numpy. It looks like you’re using Standalone on Win (so it’s the numpy version that we prepackaged)?

5

Correct. The fact that this error happens with both PTB and sounddevice is also making me think it’s something deeper, so numpy is a good candidate.

6

After some digging and adding some debug code into the sounddevice backend around the line that’s generating the error, I turned up something deeply weird. When a movie file is played on its own in a simple demo experiment, the sounddevice backend reports that self.channels = 2 and len(dat.shape) = 2. When it is played while other audio or movie files have been loaded but are not playing in this PyHab experiment, it reports that self.channels = 1. Interestingly, the line of code that runs when self.channels = 1 and len(dat.shape) = 2 is identical to when both values are 2:

toSpk[:len(dat), :] += dat

But when the program thinks self.channels = 1, this causes a crash.

So that line of code might not be appropriate in this case, but more likely the issue is in the underlying reason that the number of channels in the soundstream seems to change based on the number of things that are loaded! I’m not going to have time to work further on this for a couple days, but @jon do you have any idea what could cause that? The trace doesn’t give me a clear idea of how channels are identified or set.

7

I think for me to jump in it would help to have a minimal example including the audio/movie files to test on. Could you put on pavlovia for me to look into?

8

Finally isolated a minimal case. Super easy to replicate. This error occurs if you load a sound.Sound object, and then later load a movieStim3 object that includes sound. If you load the movieStim3 object prior to the sound object, they’re both fine.

My “demo” is literally just the MovieStim demo with two extra lines:

from psychopy import sound

demoSound = sound.Sound('someaudiofile.wav')

Just move the demoSound line above or below the line where “mov” is defined and you should get the behavior.

That at least tells me I can make a workaround, but I’m still not sure how to actually fix it.

1 Like
9

I think I’m having the same problem: Midi working, audio isn't

This is for a single participant. The experiment is working fine on most people’s computers…

Is there a simple workaround for him that doesn’t involve changing the code? Can I get him to switch the audio output of his computer to a mono output somehow? The error message seems to mention his bluetooth headphones… could these be the issue somehow?

10

Online participant or in person? This should only come up in-person.

11

‘in person’, which is to say that they’ve downloaded PsychoPy v.1.90.2 (which it’s written in) and are running it in on their own computer…

In the meantime, the problem seems to have been solved by disabling Bluetooth and changing the PsychoPy preferences accordingly. Why do you reckon a Bluetooth device should have caused an issue here?

12

Probably not the bluetooth device itself so much as the audio drivers the operating system uses to communicate with it. Especially since you were using an older version of PsychoPy I really can’t say what the specific issue was, but it sounds like you worked something out.

1 Like
13

I"m having the same issue too.

I’ve looked further into the backend code, and noticed that if set prebuffer = 0, (so that it plays from disk), than it will error out when calling self._channelCheck() .
Since the arguments for _channelCheck() is the self.sndArr. self.sndArr doesn’t get initialized uniless if prebuffer=1.

However this may or may not be a solution. Some people might not want to play the audio straight from disk.

14

I have possible solution to this.
It’s going to be frustrating but here it goes:

For all media stimulus, THE AUDIO CHANNEL HAS TO BE THE SAME.

For instance, if you’re trying to play multiple audio files, use Audacity to convert them to the same audio channel. Wether it be mono or stereo.

If you choose to use stereo, ensure that ALL media files are stereo.
If you choose to use mono, ensure that ALL media flies are mono.

From trying to go down this annoying rabbit hole, it seems to be an issue with psychotoolbox, which PsychoPy uses to process audio.

Hope this helps anyone.