Remote access (via Zoom) to touchscreen device - not registering clicks

Hi there,

I have designed an experiment where children click different “clickable” buttons on a screen to make decisions. When they make a click it takes their avatar down a certain route and they need to make further decisions.

Because of Covid, I need to run this study online but it is also protocol that an experimenter is present for the session. Therefore, my approach is to share my screen with participants via Zoom, giving them access of my screen and they can click on the buttons as if they were in the lab.

This approach works perfectly if I am sharing to participants using laptops or desktops. However, it does not recognise “clicks” when they use a touchscreen device/phone. I have checked and my exp works okay locally with a touchscreen device (although I need to double tap to register a click). But when I am giving remote control to a touchscreen device over Zoom it does not seem to register clicks at all.

I don’t mind if participants have to double tap to make a decision but I just need it to register taps at all. Has anyone else encountered a similar issue or know what the problem might be? Apologies if this has been covered already!

Thanks in advance
Matthew

OS (Win10):
**PsychoPy version - v2021 1.1.4):

1 Like

Hi Matthew,

This is actually a unique situation I have not heard of before! But I can understand why that might not work - is it essential that you allow children to perform the task using a touch screen device?). could you see if registering a “hover” on something works?

That would be implemented something like this demo:
hoverdemo.psyexp (12.5 KB)

Thanks!
Becca

Hi Becca,

Thanks for your reply. It’s not essential but I would like to give participants who may not have access to laptops/computers the opportunity to take part (although in general if people have iPads/touchscreen devices they usually have both). A “hover” mode is a great idea for touchscreen device users. I’m going to attempt to make a “touchscreen version of my task” this weekend using that if clickable.contains code you sent and I will update you if that works via Zoom.

In the meantime, I have encountered a separate issue regarding clicks, even for laptops/computers or indeed locally. Maybe I should start a different post for this (let me know if I should).

Basically, my task requires 42 trials where participants make two choices: to take a character up (to a garden) or down (to a forest) initially, then they go left or right in the garden or forest. Participants can either click or use key presses to make their decisions. To record clicks, I use the click.ispressedIn(object) code to record responses in each frame. See below for my code for left and right clicks:

if (clickScene_3.isPressedIn(keyRight_forest)) or (key_resp_10.keys == 'right'):
    forestright = 1
    forestleft = 0
elif (clickScene_3.isPressedIn(keyLeft_forest)) or (key_resp_10.keys == 'left'):
    forestleft = 1
    forestright = 0

Then in the End Routine component, if “forestleft” is 1 participants wil see the left path of the forest in the next routine etc… they are then given a reward/loss depending on trial and it subsequently loops them back to do all this again. For the most part, it works perfectly but there one or two trials in which participants click to go right and it takes them left. I have checked the files and for some of these trials in which a mistake occurs, both forestleft or forestright were set to 0 and it seems to send participants to where they last made their choice in the previous trial. Its as if the computer didn’t register the click properly so just defaults back to their last choice. Do you know what may be happening here? I have been trying to look around and I have a suspicion that perhaps the click.ispressedIn(object) code could be unreliable at times/prone to errors? Are there better alternative that would do the same thing and don’t require a lot of extra programming?

Let me know if you think I should move this qs to another thread or if I can provide anymore code that helps.

Thanks in advance,
Matthew

Hi Matthew,

Perhaps start a new thread for this one and link it here so we can continue chatting about hover responses in this thread.

For the new issue - it would be great if you could share the builder file! Out of interest does the issue occur with the keypress? I suspect at a glance you might need to use key_resp10.keys[-1] == 'right' to test if the last key pressed was “right” (the .keys attribute is usually a list if the keyboard component does not force end the routine.

Thanks,
Becca

Hi Becca,

Apologies for my very slow response to this. In the end we decided it wasn’t worth creating a version solely for touchscreen devices and instead asked participants to use a laptop/computer. Although I have no doubt the “hover” mode idea you suggested would have worked if we had more time to update the task with that!

As for my other issue, no it was to do only with clicks. What I realised was happening is that my experiment in general was using up way too much CPU and this meant that sometimes when the participant clicked on the objects psychopy didn’t record what button was clicked before it jumped ahead and ended the routine anyway. My guess is that because it didn’t know which button was clicked (forestright and forestleft were both still set to zero), so it just used the last iteration of the routine and took participants to wherever they went last time.

I managed to reduce the amount of CPU being used in my experiment by cutting down the useless variables that were being recorded and in general by streamlining code to make things a bit simpler. I also delayed the presentation of the buttons and click components so participant weren’t able to rush through the experiment and potentially causing the game to miss clicks. Additionally, I added the following code to each frame in the routines were participants made clicks:

time.sleep(0.001)

I thought this may help to delay the task for a second before executing the routine. All of this was an attempt to reduce the CPU usage which may cause unexpected delays as other processes (outside PsychoPy) finally jump in and wrest control of the CPU from you (see this forum for a discussion of this).

Finally, I also closed all other programmes on my computer when running the experiment so that all I had open was Zoom and the Psychopy experiment. All this together seemed to fix the issue as the bug never occurred again (apart from one time when I had about 20 other programmes open at once!).

Thank you for help.

Best,
Matthew