Mouse event getPressed() stops functioning

Hi all,
I am trying to make an experiment in which users press on two images in a grid of images on the screen it swaps the position of the two images. I currently have this code:

import os
from psychopy import visual, core, event

#Size of window
size = (600, 600)
#Initializes the window and clock
mywin = visual.Window(size, units='pix')

mouse = event.Mouse()
clock = core.Clock()

#loads in the images
im_dir = os.getcwd() + '/images/'
images = [im_dir + im for im in os.listdir(im_dir)
          if len(im.split('.')) == 2 and
          im.split('.')[-1] == 'jpg']

def build_grid(grid_shape, im_size, shift = (0,0)):
    """
    Gets the centers for each image such that
    the images are displayed in a grid fashion.

    First finds coordinates in standard fashion
    (top left is (0,0)) then performs a coordinate
    shift on the image center to make the grids center pixel
    (0,0).

    :param grid_shape: Tuple
        The shape of the grid
    :param im_size: Tuple
        The size of the images
        *Note all images must be of the same size!
    :param shift: (Optional) Tuple
        Location to start the grid in the window.
        *Note default is (0, 0)
    :return: List
        The centers for each image in the grid
    """
    im_center = (im_size[0]//2,im_size[1]//2)
    grid = []
    corners = []
    curr_y = shift[1]
    for j in range(grid_shape[0]):
        curr_x = shift[0]
        for i in range(grid_shape[1]):
            half_width = size[0] // 2
            half_height = size[1] // 2
            grid += [[(curr_x + im_center[0]) - half_width,
                      half_height - (curr_y + im_center[1])]]
            corners += [(grid[-1][0] - im_center[0], grid[-1][1] + im_center[1],
                          grid[-1][0] + im_center[0], grid[-1][1] - im_center[1])]
            curr_x += im_center[0] * 2
        curr_y += im_center[1] * 2
    return grid, corners

grid, corners = build_grid((4,4), (150,150), shift = (0, 0))

image_stimulus = [visual.ImageStim(win=mywin, image = images[i], units='pix', pos = grid[i])
                  for i in range(len(images))]

im1 = None
im2 = None

for frameN in range(100000):
    mouse.clickReset()
    event.clearEvents(eventType='Mouse')
    if mouse.getPressed()[0] > 0:
        x, y = mouse.getPos()
        mouse.clickReset()
        for i, (l_v, u_h, r_v, l_h) in enumerate(corners):
            if l_v < x < r_v and l_h < y < u_h:
                if im1 is None:
                    im1 = i
                elif i != im1 and im2 is None:
                    im2 = i
                while mouse.getPressed()[0] > 0: pass
                break

    if im1 is not None and im2 is not None:
        image_stimulus[im1], image_stimulus[im2] = image_stimulus[im2], image_stimulus[im1]
        image_stimulus[im2].pos, image_stimulus[im1].pos = image_stimulus[im1].pos, image_stimulus[im2].pos
        im1 = None
        im2 = None

    for im in image_stimulus: im.draw()
    mywin.flip()
    mywin.clearBuffer()

#cleanup
mywin.close()
core.quit()

*Note: I know I can optimize the coordinate search using some variation of binary search which is on my todo list. I am still in the phase of developing a better prototype so some of the code needs to be cleaned/optimized.

The issue with the above code is after around 2000 - 3000 iterations the mouse.getPressed event seems to not respond or get really slow. I have tried searching for similar issues yet I have not found any. My initial speculation was the line

while mouse.getPressed()[0] > 0: pass

was causing synching issues however when removing that line the issues still persist.

This study will be run only so I know that I will have to convert it to javascript eventually that is why I am not trying to use any other packages. I am currently running this on a MacBook Pro baseline 2019 so 8th gen 1.4 Ghz intel i5 quad core with 8 gigs of ram. Also I noticed after inspecting the activity monitor during execution around the time when the clicks become less responsive the processor usage drops to around 31% for the initial 49.5%.
Anyways any help or suggestions would be greatly appreciated. Thanks all!

If you are wanting to covert to Javascript in the long run then I’d recommend using Builder (since it has an auto translate function) and also looking at my crib sheet (because the auto translate is far from perfect).