Play sound on each click in routine

If this template helps then use it. If not then just delete and start from scratch.

OS (e.g. Win10): Win10
PsychoPy version (e.g. 1.84.x): 2022.1.3
Standard Standalone? (y/n) If not then what?: Y
What are you trying to achieve?: I would like a sound component to replay a sound with each new click in a routine.

What did you try to make it work?:
I created a sound component, a mouse component, and a clickable polygon:


The sound component plays the sound when the polygon is clicked:
image
The polygon is a clickable component for the mouse component:
image
What specifically went wrong when you tried that?: This successfully plays the sound once, but the sound does not play again upon new clicks. I believe this is because the sound component does not ā€œresetā€ on new clicks.
Include pasted full error message if possible. ā€œThat didnā€™t workā€ is not enough information.
N/A! Thanks in advance for your help.

For further detail, ideally Iā€™d like to avoid a solution involving loops if possible! However, I could potentially make use of a solution that allows the user to listen to the sound component as many times as they like before breaking out of the loop.

The purpose of the experiment is to allow the participant to listen to a few different pieces in a single routine (and at their leisure) and then compare them them with rating scales (this is why itā€™s important they can hear the pieces as many times as they like before rating).

Hi @andersoc,

Could you try setting the sound component to ā€˜set every frameā€™ instead of ā€˜set every repeatā€™ and let me know if that works?

Thanks,

Kim

Hi Kim,

Thanks for your suggestion! Unfortunately Iā€™ve tried this and it doesnā€™t seem to work.

Cam

It might be easier to create a sound in code and then you get explicit control over it, without worrying what settings are applied in the Builder sound component dialog. e.g. insert a code component on your routine (from the ā€œcustomā€ component tab) and in its ā€œbegin experimentā€ tab, put something like:

beep = sound.Sound('A', secs = 0.25) # just create this once for the whole experiment

then in its ā€œeach frameā€ tab, put something like:

if mouse.isPressedIn(polygon) and beep.status != STARTED:
    beep.play()

The second check is there so that the sound doesnā€™t keep re-starting just because the mouse button is still held down (mouse clicks arenā€™t instantaneous and could easily span more than one screen refresh). This will ensure that the beep lasts only for the specified duration. It will however play again if the mouse is held down for the entire duration of the tone - avoiding that would need another check or two to ensure that the mouse button has been released before another beep will play.

Hi Michael,

Thanks for your input! This seems to work. I slightly modified the code to enable pausing as well. I believe the elif statement is necessary for replaying within the same routine but I may be wrong.
The only minor setback of this approach is that there are some very minor hiccups upon clicking.

Thank you Michael and Kim for your help!

Cam

if mouse.isPressedIn(polygon) and piece.status != STARTED:
    piece.play()
elif mouse.isPressedIn(polygon) and piece.status == STARTED:
    piece.stop()

Edit 2022/05/20: This approach causes unwanted effects on experiments and should be avoided (see Michaelā€™s response for better solution)
ā€”

To fix the hiccuping (which occasionally leads to minor glitches when starting and stopping the audio file), adding a slight delay before and starting and stopping the audio appears to work:

if mouse.isPressedIn(polygon) and piece.status != STARTED:
    core.wait(secs = 0.25)
    piece.play()
elif mouse.isPressedIn(polygon) and piece.status == STARTED:
    core.wait(secs = 0.25)
    piece.stop()

I havenā€™t yet tested how easily this converts to PsychoJS and there is still a ā€œpopā€ after clicking, but it is a relatively minor issue

The glitch can be further mitigated by setting the argument hamming = False in the sound component.

If anyone manages to get this solution working on PsychoJS, I would be very grateful!

Thanks again

Hi, glad to see youā€™ve made this work.

But unfortunately using core.wait() is a Builder script is something to be avoided at all costs. The code that runs in the ā€œeach frameā€ tab of a code component has to do exactly that: run within one frame interval (i.e. for a typical 60 Hz display, within 16.7 ms).

By inserting a pause of 250 ms within that code, you will break Builderā€™s draw/event loop cycle. i.e. Builder updates the screen on every screen refresh, and that is when it does things like check when stimuli should be updated/stopped/started, keypresses responded to, and so on. Youā€™ve effectively frozen that process for a quarter of a second, which could have all sorts of negative consequences for the rest of the script.

So you will need something a bit more complex to achieve what you want. e.g. you could create some timers that will keep track of your click interval across screen refreshes, so you donā€™t need to pause things. e.g. at the beginning of the experiment:

play_timer = core.CountdownTimer(start = 0.25)
stop_timer = core.CountdownTimer(start = 0.25)

Then in the ā€œbegin routineā€ tab,

something like:

play_timer_started = False
stop_timer_started = False

and then modify your code to be something like:

if mouse.isPressedIn(polygon) and piece.status != STARTED:
    if not play_timer_started:
        play_timer.reset() # start counting down from 0.25 s 
        play_timer_started = TRUE
    if play_timer_started and play_timer.getTime() < 0.0: # >= 250 ms elapsed
        piece.play() 
        play_timer_started = FALSE

# and similarly for stopping

Note you need to take into account the granularity of timing. With a 60 Hz display, this code will run every 16.666 ms, or 15 times within a 250 ms period. You might want to adjust the interval to get closer to the exact time you want, as this code might trigger the sound either at 250 ms after it starts timing, or at the next refresh, 266.66 ms after it starts. Setting the timer to be, say, somewhere between 233 and 250 ms would make it more likely that it would take effect at the screen refresh that occurs 250 ms later - you can record the times to get a feeling for how accurate you are getting (if that matters).

Hi Michael,

Thank you for clarifying! I will give this a try and report back. Iā€™ll also flag in the original message the problem with my initial approach. Hopefully by then Iā€™ll have figured this out in PsychoJS too so that others might use the solution for in-person or online study.

Thanks again for your help! Here is my update to the previous post.

I realized the experiment wasnā€™t working online because I didnā€™t define the sound properly in JS:

Sound component:

Python Code (begin experiment tab):
beep = sound.Sound('A', secs = 25) # just create this once for the whole experiment
JS code:

  beep = new sound.Sound({
    win: psychoJS.window,
    value: 'A',
    secs: 10,
    });
  beep.setVolume(0.3);

I also took your suggestion and used the code you wrote for creating a timer.

Timer

Begin experiment tab:
Python

play_timer = core.CountdownTimer(start = 0.25)
stop_timer = core.CountdownTimer(start = 0.25)

JS

play_timer = new util.CountdownTimer({
    startTime: 0.25
    });
stop_timer = new util.CountdownTimer({
    startTime: 0.25
    });

Begin routine tab:
Python

play_timer_started = False
stop_timer_started = False

JS

play_timer_started = false;
stop_timer_started = false;

Finally, the code which maps the sound to the Polygon:

Click event

Python (each frame):

if mouse.isPressedIn(polygon) and beep.status != STARTED:
    if not play_timer_started:
        play_timer.reset() # start counting down from 0.25 s 
        play_timer_started = True
    if play_timer_started and play_timer.getTime() < 0.0: # >= 250 ms elapsed
        beep.play() 
        play_timer_started = False
# and similarly for stopping
elif mouse.isPressedIn(polygon) and beep.status == STARTED:
    if not play_timer_started:
        play_timer.reset() # start counting down from 0.25 s 
        play_timer_started = True
    if play_timer_started and play_timer.getTime() < 0.0: # >= 250 ms elapsed
        beep.pause() 
        play_timer_started = False


JS:

if ((mouse.isPressedIn(polygon) && (beep.status !== PsychoJS.Status.STARTED))) {
    if ((! play_timer_started)) {
        play_timer.reset();
        play_timer_started = true;
    }
    if ((play_timer_started && (play_timer.getTime() < 0.0))) {
        beep.play();
        play_timer_started = false;
    }
} else {
    if ((mouse.isPressedIn(polygon) && (beep.status === PsychoJS.Status.STARTED))) {
        if ((! play_timer_started)) {
            play_timer.reset();
            play_timer_started = true;
        }
        if ((play_timer_started && (play_timer.getTime() < 0.0))) {
            beep.pause();
            play_timer_started = false;
        }
    }
}

This solution works excellently in Builder but does not work in PsychoJS (possibly due to a translation error on my end).
The following code does work in PsychoJS but has the same hiccuping issue. This involves defining the sound and click event, but without any timer to add a delay to when the tone onset occurs.

Click event

Python (each frame):

if mouse.isPressedIn(polygon) and beep.status != STARTED:
    beep.play()
elif mouse.isPressedIn(polygon) and beep.status == STARTED:
    beep.stop()

JS:

if ((mouse.isPressedIn(polygon) && (beep.status !== PsychoJS.Status.STARTED))) {
    beep.play();
} else {
    if ((mouse.isPressedIn(polygon) && (beep.status === PsychoJS.Status.STARTED))) {
        beep.stop();
    }
}

As such, I think the issue may have to do with how I defined the timer. I used the following documentation as a reference: JSDoc: Class: CountdownTimer

I will keep trying over the next week and will report any updates.

1 Like

Good luck. Iā€™m no help on the JavaScript side of things - if you donā€™t solve this yourself, hopefully one of the Open Science Tools team can jump in.

1 Like

Thanks for all your help in any case! You helped me to get most of the way there :slight_smile: