Midi works! However....making a class for using midi

Thanks for your help so far (Doesn't play midi files).

So, it turns you can play midi files (.mid) in PsychoPy using pygame (see below), However, if you just set pygame first in audio library and call a file as follows…

from psychopy import sound
audio_stimulus = sound.Sound('/Users/sam/Downloads/Experiment/sounds/Sound1.mid', secs=-1)
audio_stimulus.play()

…then you just hear a click and receive the following feedback:

<psychopy.sound.backend_pygame.SoundPygame object at 0x112edcb10>

However if you call a midi file like this:

import pygame
pygame.mixer.music.load("/Users/sam/Downloads/Experiment/sounds/Sound1.mid")
pygame.mixer.music.play("/Users/sam/Downloads/Experiment/sounds/Sound1.mid")

Then it plays fine.

I presume that anything from these links will work:
http://www.pygame.org/docs/ref/music.html
http://www.pygame.org/docs/ref/midi.html

The thing is, in my code (which presently calls .wav files) I currently use all of the following, which I’ve inherited from the builder-generated code:

from psychopy import sound
audio_stimulus = sound.Sound('A', secs=-1)
audio_stimulus.setVolume(1)
audio_stimulus.setSound(audioStim, secs=-1)
audio_stimulus.tStart = t
audio_stimulus.frameNStart = frameN 
audio_stimulus.play() 
audio_stimulus.stop()
audio_stimulus.status

Could someone smarter than me help me to write a class that I could seamlessly set audio_stimulus to, that won’t otherwise affect how my code runs (i.e. it will also need to have the statuses (NOT_STARTED, STARTED, PLAYING, PAUSED,STOPPED, FINISHED, PRESSED, RELEASED, FOREVER), setSound() using the appropriate column of the conditions excel file etc…

I note that pygame has these options already:

pygame.mixer.music.set_volume(1)
pygame.mixer.music.get_busy # this checks if the music is still running
pygame.mixer.music.stop()
pygame.mixer.music.play()

Sorry if this seems lazy… basically out of interest I had a look at the code for backend_pyo.py and it all looks incredibly complicated. Seems more sensible to ask if any of you can help me, before embarking on something like that…

Here’s the class creation for Sound using pyo (I think)…

class SoundPyo(_SoundBase):

    def __init__(self, value="C", secs=0.5, octave=4, stereo=True,
                 volume=1.0, loops=0, sampleRate=44100, bits=16,
                 hamming=True, start=0, stop=-1,
                 name='', autoLog=True):
        """
        value: can be a number, string or an array:
            * If it's a number between 37 and 32767 then a tone will be
              generated at that frequency in Hz.
            * It could be a string for a note ('A', 'Bfl', 'B', 'C',
              'Csh', ...). Then you may want to specify which octave as well
            * Or a string could represent a filename in the current location,
              or mediaLocation, or a full path combo
            * Or by giving an Nx2 numpy array of floats (-1:1) you can
              specify the sound yourself as a waveform

            By default, a Hamming window (5ms duration) will be applied to a
            generated tone, so that onset and offset are smoother (to avoid
            clicking). To disable the Hamming window, set `hamming=False`.

        secs:
            Duration of a tone. Not used for sounds from a file.

        start : float
            Where to start playing a sound file;
            default = 0s (start of the file).

        stop : float
            Where to stop playing a sound file; default = end of file.

        octave: is only relevant if the value is a note name.
            Middle octave of a piano is 4. Most computers won't
            output sounds in the bottom octave (1) and the top
            octave (8) is generally painful

        stereo: True (= default, two channels left and right),
            False (one channel)

        volume: loudness to play the sound, from 0.0 (silent) to 1.0 (max).
            Adjustments are not possible during playback, only before.

        loops : int
            How many times to repeat the sound after it plays once. If
            `loops` == -1, the sound will repeat indefinitely until stopped.

        sampleRate (= 44100): if the psychopy.sound.init() function has been
            called or if another sound has already been created then this
            argument will be ignored and the previous setting will be used

        bits: has no effect for the pyo backend

        hamming: whether to apply a Hamming window (5ms) for generated tones.
            Not applied to sounds from files.
        """
        global pyoSndServer
        if pyoSndServer is None or pyoSndServer.getIsBooted() == 0:
            init(rate=sampleRate)

        self.sampleRate = pyoSndServer.getSamplingRate()
        self.format = bits
        self.isStereo = stereo
        self.channels = 1 + int(stereo)
        self.secs = secs
        self.startTime = start
        self.stopTime = stop
        self.autoLog = autoLog
        self.name = name

        # try to create sound; set volume and loop before setSound (else
        # needsUpdate=True)
        self._snd = None
        self.volume = min(1.0, max(0.0, volume))
        # distinguish the loops requested from loops actual because of
        # infinite tones (which have many loops but none requested)
        # -1 for infinite or a number of loops
        self.requestedLoops = self.loops = int(loops)

        self.setSound(value=value, secs=secs, octave=octave, hamming=hamming)
        self.needsUpdate = False

    def play(self, loops=None, autoStop=True, log=True):
        """Starts playing the sound on an available channel.

        loops : int
            (same as above)

        For playing a sound file, you cannot specify the start and stop
        times when playing the sound, only when creating the sound initially.

        Playing a sound runs in a separate thread i.e. your code won't wait
        for the sound to finish before continuing. To pause while playing,
        you need to use a `psychopy.core.wait(mySound.getDuration())`.
        If you call `play()` while something is already playing the sounds
        will be played over each other.
        """
        if loops is not None and self.loops != loops:
            self.setLoops(loops)
        if self.needsUpdate:
            # ~0.00015s, regardless of the size of self._sndTable
            self._updateSnd()
        self._snd.out()
        self.status = STARTED
        if autoStop or self.loops != 0:
            # pyo looping is boolean: loop forever or not at all
            # so track requested loops using time; limitations: not
            # sample-accurate
            if self.loops >= 0:
                duration = self.getDuration() * (self.loops + 1)
            else:
                duration = FOREVER
            self.terminator = threading.Timer(duration, self._onEOS)
            self.terminator.start()
        if log and self.autoLog:
            logging.exp("Sound %s started" % (self.name), obj=self)
        return self

    def _onEOS(self):
        # call _onEOS from a thread based on time, enables loop termination
        if self.loops != 0:  # then its looping forever as a pyo object
            self._snd.stop()
        if self.status != NOT_STARTED:
            # in case of multiple successive trials
            self.status = FINISHED
        return True

    def stop(self, log=True):
        """Stops the sound immediately"""
        self._snd.stop()
        try:
            self.terminator.cancel()
        except Exception:  # pragma: no cover
            pass
        self.status = STOPPED
        if log and self.autoLog:
            logging.exp("Sound %s stopped" % (self.name), obj=self)

    def _updateSnd(self):
        self.needsUpdate = False
        doLoop = bool(self.loops != 0)  # if True, end it via threading.Timer
        self._snd = pyo.TableRead(self._sndTable,
                                  freq=self._sndTable.getRate(),
                                  loop=doLoop, mul=self.volume)

    def _setSndFromFile(self, fileName):
        # want mono sound file played to both speakers, not just left / 0
        self.fileName = fileName
        self._sndTable = pyo.SndTable(initchnls=self.channels)
        # in case a tone with inf loops had been used before
        self.loops = self.requestedLoops
        # mono file loaded to all chnls:
        try:
            self._sndTable.setSound(self.fileName,
                                    start=self.startTime, stop=self.stopTime)
        except Exception:
            msg = ('Could not open sound file `%s` using pyo; not found '
                   'or format not supported.')
            logging.error(msg % fileName)
            raise TypeError(msg % fileName)
        self._updateSnd()
        self.duration = self._sndTable.getDur()

    def _setSndFromArray(self, thisArray):
        self._sndTable = pyo.DataTable(size=len(thisArray),
                                       init=thisArray.T.tolist(),
                                       chnls=self.channels)
        self._updateSnd()
        # a DataTable has no .getDur() method, so just store the duration:
        self.duration = float(len(thisArray)) / self.sampleRate

It looks as thought the PsychoPy statuses are set up in constants.py

from __future__ import absolute_import, print_function

import sys
if sys.version_info.major >= 3:
    PY3 = True
else:
    PY3 = False

NOT_STARTED = 0
PLAYING = 1
STARTED = PLAYING
PAUSED = 2
STOPPED = -1
FINISHED = STOPPED

# for button box:
PRESSED = 1
RELEASED = -1

# while t < FOREVER ... -- in scripts generated by Builder
FOREVER = 1000000000  # seconds

The code for creating the SoundPyo class seems massively complicated (to my ignorant eyes anyhow…)… surely for my purposes (see original post) setting the statuses needn’t be so complicated? :fearful:

Any help gratefully received :slight_smile:

In terms of the standard builder code, I’m correct in thinking that tStart and frameNStart only make it to the data files if they have come from the keyboard response, aren’t I? In which case perhaps they’re not actually that important here? Is there actually any need to save these aspects for anything apart from the keyboard response?

So yeah if it’s possible to make a relatively straightforward class that covers my need then let me know! By the same token, if I need to have a rethink then let me know this also… Using midi files, rather than wav, would really really help this experiment.

As my ‘I don’t really know what I’m doing’ attempt, I came up with this:

from psychopy.constants import (STARTED, PLAYING, PAUSED, FINISHED, STOPPED,
                                NOT_STARTED, FOREVER)
import pygame

class midiPlayer(object):
    status = NOT_STARTED
    tStart = 0
    frameNStart = 0
    setVolume = 1
    def __init__(self, Sound):
        self.Sound = Sound 
    def play(self):
        freq = 44100    # audio CD quality
        bitsize = -16   # unsigned 16 bit
        channels = 2    # 1 is mono, 2 is stereo
        buffer = 1024    # number of samples
        pygame.mixer.init(freq, bitsize, channels, buffer)
        pygame.mixer.music.set_volume(self.setVolume)
        pygame.mixer.music.load(self.Sound)
        pygame.mixer.music.play()
        self.status = STARTED
    def stop(self):
        pygame.mixer.music.stop()
        self.status = FINISHED
    def setSound(self, music_file):
        self.Sound = music_file

What problems can you see with this in terms of using it in replacement of the Psychopy in house ‘sound’ (I’m sure there are many!) ??

Good luck with this. Pygame had the least good timing of all the sound engines so I’ve not gone near it in a long time.

Most of those attributes are there for a generality purpose. If the user wanted a stimulus to have a certain duration (rather than certain end time then we need to know what time it started etc. Even things like status updates are only needed for Builder purposes (because it has to make a decision on each frame abot whether we need to change the stimulus.

If you’re controlling timing etc from your own script then your basic class is probably fine.

Thanks @jon

I’ve updated the class to something like this:

class midiPlayer(object):
    status = NOT_STARTED
    tStart = 0
    frameNStart = 0
    setVolume = 1
    def __init__(self, Sound):
        self.Sound = Sound 
    def play(self):
        freq = 44100    # audio CD quality
        bitsize = -16   # unsigned 16 bit
        channels = 2    # 1 is mono, 2 is stereo
        buffer = 1024    # number of samples
        pygame.mixer.init(freq, bitsize, channels, buffer)
        pygame.mixer.music.set_volume(self.setVolume)
        pygame.mixer.music.load(self.Sound)
        pygame.mixer.music.play()
        self.status = STARTED 
    def stop(self):
        pygame.mixer.music.stop()
        self.status = FINISHED
    def setSound(self, music_file):
        self.Sound = music_file
    def busy(self):
        return pygame.mixer.music.get_busy() 

As you say, I don’t actually need tStart etc. I’ve just left them in anyway but I’m not using them. midiPlayer.busy() checks if the music is still playing.

Currently I’m needing to explicitly set audio.status to FINISHED once it’s finished playing. As such I currently have a bit of code that looks like this:

if audio_feedback.status == STARTED and audio_feedback.busy() == 0:
    audio_feedback.status = FINISHED

Although I’m worried that running the audio_feedback.busy() function like this on every frame might be a strain on the system? If that’s the case, is there a smarter way in which I can do this?

The memory usage does indeed seem to be steadily rising again (Memory leak - what could _LogEntry be?) now that I’ve made these changes to use midi (first time around in turned out that I was making a new text stimulus on every frame… :see_no_evil:), although this isn’t as steep a rise as it was first time. Do you reckon this might be because I keep running the audio_feedback.busy() function, or would you predict some other reason?

Thanks :slight_smile:

I’m afraid I don’t know anything about the pygame mixer these days - last time I used it there was no such busy() call. All you can do is test it with and without.

When you say the memory increases, is that with each sound stimulus you create, or on each time you call play() or during playback with a single play() call?