Mouse click moves to next trial instead of dragging stimuli

Josh_Kazi · February 20, 2021, 6:54pm

Hi, I am new to psychopy and python, and am having drag issues with the click and drag function for my experiment. My experiment consists of a few still stimuli and a movable stimuli that is placed at a random position at the beginning of each trial. The goal of the experiment is for the user to align the movable stimuli with what they believe to be the center of the still stimuli. This repeats 4 more times (still stimuli move closer to the origin with each trial). I intended for the moving stimuli to be clicked and dragged with the left mouse button, and when the user thinks the stimuli are aligned, they move on to the next trial with a right mouse click. I have implemented this, however, I noticed that there is a significant lag with the mouse when the stimuli is dragged too fast. I consulted this forum post to fix this, and wrote my code as such:

# ------Begin Routine-------
        continueRoutine = True
        # update component parameters for each repeat
        # setup some python lists for storing info about the mouse
        gotValidClick = False  # until a click is received
        # keep track of which components have finished
        
        
        if expInfo['Direction'] == 'Horizontal':
            TrialPhaseComponents = [polygon, StillStimuli3, StillStimuli4, mouse]
            
        for thisComponent in TrialPhaseComponents:
            thisComponent.tStart = None
            thisComponent.tStop = None
            thisComponent.tStartRefresh = None
            thisComponent.tStopRefresh = None
            if hasattr(thisComponent, 'status'):
                thisComponent.status = NOT_STARTED
        # reset timers
        t = 0
        _timeToFirstFrame = win.getFutureFlipTime(clock="now")
        TrialPhaseClock.reset(-_timeToFirstFrame)  # t0 is time of first possible flip
        frameN = -1

and:

        # -------Frame Code-------
        while continueRoutine:
            # get current time
            t = TrialPhaseClock.getTime()
            tThisFlip = win.getFutureFlipTime(clock=TrialPhaseClock)
            tThisFlipGlobal = win.getFutureFlipTime(clock=None)
            frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
            # update/draw components on each frame
            
            drag_in_process = False 
                
            if not drag_in_process: # check if one should be
                    if mouse.isPressedIn(polygon):
                        clicked_stimulus = polygon
                        drag_in_process = True
                        break
            
            if True in mouse.getPressed():
                if drag_in_process:
                    if mouse.isPressedIn(polygon, buttons=[0]): #left mouse button moves polygon
                        continueRoutine = True  #left mouse does not advance trial
                        clicked_stimulus.pos = mouse.getPos()
                        
                    elif mouse.isPressedIn(polygon, buttons=[2]): #right mouse button moves to next routine
                        continueRoutine = False #right mouse advances trial
                else:
                    drag_in_process = False

As the title suggests, implementing this code now moves the experiment to a new trial with any click on the moving stimuli, instead of being able to drag and click the right mouse button, much less fix the drag lag. I tried multiple variations of the frame code but still no luck. Below I have attached a simplified, working version of my experiment before I tried to implement the lag “fix”. Any help would be greatly appreciated.

test.py (15.2 KB)

Michael · February 21, 2021, 7:13pm

It’s easier for us to read your .psyexp file than to wade through an entire .py file. Just to check though: are you entering code through a code component in the Builder view or are you attempting to edit the .py script? We really don’t recommend the latter approach, precisely because it can lead to issues like this.

Josh_Kazi · February 20, 2021, 7:22pm

Hi Michael,

My apologies, I was attempting to edit the python script from coder view. I do not have a .psyexp version of the file as I am continuing this project from a previous author, whom has only given me a .py to work from.

Michael · February 23, 2021, 12:03am

OK, that’s understandable.

Notice that in the line after if not drag in process: there is a double level of indentation? That came from deleting the line in the original post for stimulus in stimuli:, which would loop through a list of stimuli. The line break exits that loop upon detecting a mouse click.

I guess you deleted the for loop because you only have one object (polygon) to track, but if so, the break also needs to be deleted. In this case, it will break the next higher loop, which we can’t see in this code snippet but it might be what is causing the routine to stop prematurely.

So in a first attempt to fix things, amend that code to read:

            if not drag_in_process: # check if one should be
                if mouse.isPressedIn(polygon):
                    clicked_stimulus = polygon
                    drag_in_process = True

I’m guessing you could also delete clicked_stimulus = polygon, as you only have one object that could be clicked.

You should do this on every frame: drag_in_process = False , as it will immediately undo the tracking of any drag. That line should only appear at the beginning of the routine (this is why we recommend the use of code components: they make it easy by inserting the relevant code at the correct point in the timeline).

You can also delete continueRoutine = True. That variable always begins the routine set to True. Action only occurs by setting it to be False.

Lastly, if you are wanting to avoid the drag-lag issue, then I think lines like this will cause you problems:

if mouse.isPressedIn(polygon, buttons=[0]):

because they pre-suppose that the mouse will stay within the bounds of the stimulus. If this was the case, we wouldn’t need to keep track of drag_in_process. So I think you just need to check the state of the mouse buttons, without also applying the constraint of being within the bounds of the stimulus. e.g.

if mouse.getPressed[0]:

There also seemed to be other issues with the indentation in the code (indentation is semantic in Python). So it should look something like this:

            if not drag_in_process: # check if one should be
                if mouse.isPressedIn(polygon):
                    drag_in_process = True
            
            if True in mouse.getPressed():
                if drag_in_process and mouse.getPressed[0]: #left mouse button moves polygon
                    polygon.pos = mouse.getPos()
                        
                if mouse.getPressed[2]: #right mouse button moves to next routine
                    continueRoutine = False #right mouse advances trial
            else:
                drag_in_process = False

Josh_Kazi · February 22, 2021, 9:11pm

Hi, I have tried using your fix below as such:

        # ------Prepare to start Routine "TrialPhase"-------
        drag_in_process = False 
        continueRoutine = True
        # update component parameters for each repeat
        # setup some python lists for storing info about the mouse
        gotValidClick = False  # until a click is received
        # keep track of which components have finished
        
        
        if expInfo['Direction'] == 'Horizontal':
            TrialPhaseComponents = [polygon, StillStimuli3, StillStimuli4, mouse]
            
        for thisComponent in TrialPhaseComponents:
            thisComponent.tStart = None
            thisComponent.tStop = None
            thisComponent.tStartRefresh = None
            thisComponent.tStopRefresh = None
            if hasattr(thisComponent, 'status'):
                thisComponent.status = NOT_STARTED
        # reset timers
        t = 0
        _timeToFirstFrame = win.getFutureFlipTime(clock="now")
        TrialPhaseClock.reset(-_timeToFirstFrame)  # t0 is time of first possible flip
        frameN = -1
        
        # -------Run Routine "TrialPhase"-------
        while continueRoutine:
            # get current time
            t = TrialPhaseClock.getTime()
            tThisFlip = win.getFutureFlipTime(clock=TrialPhaseClock)
            tThisFlipGlobal = win.getFutureFlipTime(clock=None)
            frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
            # update/draw components on each frame
            
            if not drag_in_process: # check if one should be
                if mouse.isPressedIn(polygon):
                    drag_in_process = True
            
            if True in mouse.getPressed():
                if drag_in_process and mouse.getPressed[0]: #left mouse button moves polygon
                    polygon.pos = mouse.getPos()
                        
                if mouse.getPressedIn[2]: #right mouse button moves to next routine
                    continueRoutine = False #right mouse advances trial
            else:
                drag_in_process = False

But now the stimuli does not respond to any mouse clicks at all, and is frozen on the screen. Any reason for this?

Michael · February 23, 2021, 12:16am

I gave you a typo in:

if mouse.getPressedIn[2]:

It should be:

if mouse.getPressed()[2]:

and there should be () brackets after each function call, so amend to something like this:

            if not drag_in_process: # check if one should be
                if mouse.isPressedIn(polygon):
                    drag_in_process = True
            
            if True in mouse.getPressed():
                if drag_in_process and mouse.getPressed()[0]: #left mouse button moves polygon
                    polygon.pos = mouse.getPos()
                        
                if mouse.getPressed()[2]: #right mouse button moves to next routine
                    continueRoutine = False #right mouse advances trial
            else:
                drag_in_process = False

But those typos should have given you an explicit error. Do you see any error messages, either in the Runner window, or in the pane at the bottom of the Coder window?

Otherwise, we need to check what the mouse buttons return when we check them. Before the current code, insert something like this:

# temporary debugging:
print(mouse.getPressed())
print(drag_in_process)

if not drag_in_process:
    # etc etc

and show us some of the output. It may be easiest if you set the experiment to run in a window rather than full-screen, so you can see the real-time output of those (temporary) print statements.

Josh_Kazi · February 23, 2021, 1:24am

Yes, that seemed to fix the issue, thank you. The drag lag seems to be completely fixed; the error previously was “TypeError: ‘method’ object is not subscriptable”. Although now that the drag lag is fixed, a new issue has seemed to appear. The experiment prematurely skips to the final trial after pressing the right mouse button twice instead of running through 5 trials.

Michael · February 23, 2021, 5:11am

OK, that is because the code runs too quickly, as several screen refreshes can easily occur between the button being pressed and subsequently released. So we need to ensure that we only respond once to each right mouse button click. I’m sitting outside watching a match between the England and New Zealand women’s cricket teams so bear with me while I try to type some revised code on a phone in bright sunshine.

Michael · February 23, 2021, 5:38am

OK, barring a spectacular batting collapse from England, I think we can call this early in the second innings as a likely win for the visitors.

So here goes:

            if True in mouse.getPressed():
                if drag_in_process and mouse.getPressed()[0]: # left mouse button moves polygon
                    polygon.pos = mouse.getPos()
                        
                if mouse.getPressed()[2] and not right_button_down: # right mouse button moves to next routine
                    continueRoutine = False # right mouse advances trial
                    right_button_down = True # keep track to respond only once per click
            else:
                drag_in_process = False  
                right_button_down = False

To be safe, add this line to the “begin experiment” tab:

right_button_down = False

Battle test that with lots of mouse click sequences and let me know if there are further problems.

Josh_Kazi · February 23, 2021, 5:58pm

Hey Michael, sorry to hear about the game.

I have implemented your code as well as the line to the begin experiment tab. The issue now is that the right mouse button clicks do not register at all. Just to make sure my formatting is correct, I’ll send how I implemented it:

        # ------Prepare to start Routine "TrialPhase"-------
        right_button_down = False
        continueRoutine = True
        # update component parameters for each repeat
        # setup some python lists for storing info about the mouse
        gotValidClick = False  # until a click is received
        # keep track of which components have finished
        
        if expInfo['Direction'] == 'Vertical':
            TrialPhaseComponents = [polygon, StillStimuli1, StillStimuli2, mouse]
        elif expInfo['Direction'] == 'Horizontal':
            TrialPhaseComponents = [polygon, StillStimuli3, StillStimuli4, mouse]
        elif expInfo['Direction'] == 'Both':
            TrialPhaseComponents = [polygon, StillStimuli1, StillStimuli2, StillStimuli3, StillStimuli4, mouse]
            
            
        for thisComponent in TrialPhaseComponents:
            thisComponent.tStart = None
            thisComponent.tStop = None
            thisComponent.tStartRefresh = None
            thisComponent.tStopRefresh = None
            if hasattr(thisComponent, 'status'):
                thisComponent.status = NOT_STARTED
        # reset timers
        t = 0
        _timeToFirstFrame = win.getFutureFlipTime(clock="now")
        TrialPhaseClock.reset(-_timeToFirstFrame)  # t0 is time of first possible flip
        frameN = -1
        drag_in_process = False 
        # -------Run Routine "TrialPhase"-------
        while continueRoutine:
            # get current time
            t = TrialPhaseClock.getTime()
            tThisFlip = win.getFutureFlipTime(clock=TrialPhaseClock)
            tThisFlipGlobal = win.getFutureFlipTime(clock=None)
            frameN = frameN + 1  # number of completed frames (so 0 is the first frame)
            # update/draw components on each frame
            
            if not drag_in_process: # check if one should be
                if mouse.isPressedIn(polygon):
                    drag_in_process = True
            
            if True in mouse.getPressed():
                if drag_in_process and mouse.getPressed()[0]: # left mouse button moves polygon
                    polygon.pos = mouse.getPos()
                        
                if mouse.getPressed()[2] and not right_button_down: # right mouse button moves to next routine
                    continueRoutine = False # right mouse advances trial
                    right_button_down = True # keep track to respond only once per click
            else:
                drag_in_process = False  
                right_button_down = False

Thank you again for your continued help.

Michael · February 23, 2021, 9:59pm

That line ^^^ can’t be at the beginning of the routine, or else it will stop us keeping track of the right mouse button presses across routine boundaries, by resetting it all the time, regardless of whether the mouse buttons has been released. As above, it only needs to be initialised once, in the “Begin experiment” tab.

Josh_Kazi · February 23, 2021, 11:48pm

Sorry if this seems like a really obvious thing that I should know, but where would the begin experiment tab be located? I know that it’s easy to identify on builder view but I am having a hard time finding it within my code, as I have placed right_button_down = False in multiple locations with no avail. Again, thank you.

Michael · February 24, 2021, 12:40am

Sorry, I keep forgetting you’re not using Builder (which makes this stuff easy).

Looking at the test.py script you posted above, there seem to be some statements around lines 139-140 where your script seems to define some custom variables:

height_moa = 10

polyHeight = int(expInfo['Distance from Screen(cms)']) * math.tan(height_moa * math.pi /60/180)

So putting it just before or after those lines would seem to be a natural place, before any of the loops and routines start.

Josh_Kazi · February 24, 2021, 1:19am

Okay, I tried putting it before and after both lines, but unfortunately the same issue still persists and the right mouse button does not move the trials forward. I also tried some other places around what I believed to be before the routine, and it seems to just not respond to the right click at all.

Michael · February 24, 2021, 2:03am

OK, that makes me think that line might now be in multiple places somehow, and breaking the logic.

I think you need to go back to the temporary debugging approach suggested above and put in some print statements to give real-time feedback on what code is executed when each mouse button is pressed.