Do not register touch while playing a recording

Hi,
In my experiment, the participants were asked to listen to a story and answer some questions afterwards.
The participants were asked to press a button when they were ready to answer the questions or press another button if they wanted to listen again. The experiment ran smoothly on new Surface Pro and touches were registered as mouse clicks.
Because the subjects were mostly children (< 11-year-old), I created a function to prevent them from skipping the story. It looks like this:

# create a function to play audio
def presentAudio(file, time=0):
    audio = Sound(file,sampleRate=48000,stereo=True,secs=0.01)
    audio.play()
    if time <= 0:
        wait(audio.getDuration()+0.5)
    else:
        wait(time)
    audio.stop()

As a result, the program would (presumedly) ignore any touches while playing the recording.
But when the subject kept touching on the screen (some of the children did so despite the fact that the experimenter told him to stop), python would crash.
I’m wondering if there is a possible way to tell python to ignore touches while playing the recording? Or if there’s a better way to achieve the same goal?
Any suggestion is much appreciated!

Thanks in advance!

Here is my guess what happens:
Due to using “wait”, Windows cannot dispatch the touch event to your PsychoPy window. Thus, windows assumes that the window is unresponsive and offers the possibility to close your PsychoPy experiment.

If I am correct, replacing the “wait” code with a while loop flipping the window until the audio file is finished should solve the problem.

Try this:

REPLACE

    if time <= 0:
        wait(audio.getDuration()+0.5)
    else:
        wait(time)

WITH

    if time <= 0:
        time = audio.getDuration()+0.5
    
    timer = core.CountdownTimer(time)
    while timer.getTime() > 0:  
        win.flip()

–> note that you need to pass the reference to your window to your function, in my case called “win”

Just a side note:
Is your experiment detecting all touches as mouse click events? In my case, only touches including a small swipe were detected as click events, but not a touch on the screen without finger motion.

Just in case your are facing the same issue, here is the solution I developed to deal with this issue:

# How to use:
#
# Replace old code, such as
# my_mouse = event.Mouse()
#
# with the complete following code:

# Ensures that also touch events without swipe motion are registered as mouse clicks in PsychoPy (tested with Surface Pro Tablets)
# Note:
# - short touch: left mouse button press, long touch: right mouse button press
# - ensure to check for mouse clicks after each win.flip(). Best practice: call win.flip() only once at the end of each trial processing loop -> Otherwise you might miss touch events without swipe motion
# Author: Frank Papenmeier, frank.papenmeier@uni-tuebingen.de, Oct 12 2017
import types
class TouchMouse(event.Mouse):
    def __init__(self, visible=True, newPos=None, win=None):
        event.Mouse.__init__(self, visible, newPos, win)
        self._last_frames = 0
        self._updatePressed() # call once to initialize self._buttons
        self.win.flip = types.MethodType(self._flip_and_update, self.win.flip) # update pressed state after each flip to ensure that pressed state stays current throughout each flip cycle
        
    def _flip_and_update(self, win_flip):
        # Update pressed status only once per window flip
        win_flip()
        self._updatePressed()
        
    def _updatePressed(self):
        buttons, times = event.Mouse.getPressed(self, True)
        
        self._buttons = buttons[:]
        
        if hasattr(self, '_times'): # available beginning with second pass
            for i in range(len(self._buttons)):
                if times[i] != self._times[i]: # self._times are times of last frame; thus: was there a change in times? If yes, then there was a touch event
                    self._buttons[i] = 1
            
        self._times = times[:] # make copy in order to save old state (otherwise it is always identical with original times)
    
    def getPressed(self, getTime=False):
        if getTime:
            return self._buttons, self._times
        else:
            return self._buttons
            
    def clickReset(self, buttons=(0, 1, 2)):
        event.Mouse.clickReset(self, buttons)
        self._times = event.mouseTimes[:] # prevent fake click that would otherwise occur after calling clickReset

my_mouse = TouchMouse()
3 Likes

Brilliant! It works as magic.

Yes, in my experiment touches are only detected as clicks with a small swipe.
I tried to deal with it but eventually gave up :pensive:
I modified your code and it works on new Surface Pro too! :tada:
Thank you so much!

To use isPressedIn() with TouchMouse, one should autoDraw everything and wrap win.flip() inside a while loop.
A working example:

from psychopy.visual import Window, ImageStim
from TouchMouse import TouchMouse

# create a Window
win = Window(size=(1920, 1080), units='pix', fullscr=True)
my_mouse = TouchMouse()
# draw a image
ok = ImageStim(win, image='ok.png')
ok.draw()
win.flip()
ok.autoDraw = True
# wait until it's pressed
while True:
    win.flip()
    if my_mouse.isPressedIn(shape=ok):
        # prevent it from reappearing
        ok.autoDraw = False
        break

# close the Window
win.close()

Hope this thread helps people interested in working with touchscreen!

(ok.png could be download below)

1 Like

Great to hear that the code helped you.

Just wondering: Was it necessary to re-define isPressedIn and getPos within TouchMouse? As TouchMouse is a child of event.Mouse, I assume that these functions should work out of the box without a need of explicitly adding them to TouchMouse.

Hi,
I tried to use isPressedIn after importing your scripts. But the full script couldn’t run without problems.
It seemed that the original getPos did not work with TouchMouse. As a result, I tried to re-define the methods explicitly and it works as expected.
I wonder if you are able to use isPressedIn successfully with TouchMouse?

My doctoral student is using the code and she said that isPressedIn worked for her without any problems.

I tested it again and it works.
I think other errors occurred when I used the function and I misattributed the errors.
Thanks for the clarification! I will edit the previous reply.

TouchMouse class works perfectly fine with PsychoPy3 v3.1.5. I tested this with Samsung touch-screen monitor. Using the mouse object didn’t catch the touch response before, so I added a code snippet using TouchMouse and it’s working great.

Thank you for sharing your code. I do appreciate your hard work!

I was contacted by mail whether it would be possible to use this code also in Builder view. As I was curious, I developed the attached hack to use it with Builder view also.

The basic idea: First add the mouse component and after the mouse component insert a code component which replaces the original mouse with the TouchMouse. This solution is a little hacky, especially as I used the three lines added by mouse component and modified them to work with TouchMouse. So it might break with future version of PsychoPy.

I did not test it with a touchscreen personally, but I was told that it worked as expected. Thus, I thought that I should share it also here in case someone else might need this solution in the future:

touch-mouse-psychopy-test-hack-also-builder.psyexp (8.6 KB)

1 Like

I can confirm, it works perfectly in Builder (not tested online yet). Thank you so much!!