Generating stimuli in response to subject actions

Hi all,

This is my first post about my first project using PsychoPy. I’m as newbie as they come and I look forward to getting to know PsychoPy and the community better :slight_smile:

I want to put together an experiment with the following procedure:

  1. Present subject with an image and instruct them to click on points (no restrictions on the number of points) of the image using a mouse for a set time, say 90 seconds.
  2. Each time the subject clicks a point:
    i. The position is registered in the experiment data
    ii. All subsequent presentations of the image will include a cross marking the point that the participant clicked.
  3. For a “cooldown period” of 0.5 seconds, participant clicks will not be registered.
  4. Until the 90s have passed, this loop of registering clicks, generating crosses on the image and “cooling down” continues.

There will be a limited number of areas which indicate “correct” responses, but the participant will not be given feedback on whether or not their response was correct or not. The only feedback will be the crosses demarcating the points that the subject has clicked, as a kind of memory aid. Just as there is no limit to the number of responses the subject can give during the 90s, there is no limit for the number of crosses that are generated/displayed.

I’m in the very early stages of even trying to figure out PsychoPy so there are obviously many things I need to do, but mostly I have a rough idea of how they might be solved. When it comes to the part about generating the crosses based on subject actions however, I don’t really know where to even begin.

I haven’t found anything in the documentation for mouse events( Mouse Component — PsychoPy v2023.2.3 and psychopy.event - for keypresses and mouse clicks — PsychoPy v2023.2.3 ).

I searched the forums but the things that turned up weren’t quite the same, e. g.:

The main difference is that the threads I saw talked about modifying stimuli in response to subject actions, but not generating novel (or at least, new copies of) stimuli in response to actions.

Does anyone have tips for me on this? If I need to jump into Python coding to achieve the intended effect then that’s fine since I know Python from before, I just need some pointers on how to start working on this kind of reactivity in PsychoPy.

(I tried creating a very, very basic illustration of what I’m getting at and uploaded it on gitlab. I haven’t used gitlab before either, but I hope the link works: No repository · Lowe Wilsson / Responsive clickable images project · GitLab )

Hi @Arboc, this will get you started with creating crosses on clicks. In this example, using Builder I create a polygon, a mouse and a code component. When the polygon is clicked, the makeCross function is called, which creates a small cross at the location of the mouse click, draws it, and returns the cross object. The returned cross object is stored in a list for later use. You will need the cross objects stored so that you can get the positions of the crosses, and so you can stop drawing them at the end of the trial (if that is what you want to do).

# Begin Experiment
def makeCross(pos):
    cross = visual.ShapeStim(win, pos=pos, depth=-2, vertices='cross', size=(.01,.01), fillColor="black")
    cross.setAutoDraw(True)
    return cross

# Begin Routine
crosses = []  # list container for storing cross objects

# Each Frame
if mouse.isPressedIn(polygon):
    crosses.append(makeCross(mouse.getPos()))
2 Likes

Great, that helps a lot! I’ll try to implement it right away, if I find anything interesting I’ll be sure to post it here :slight_smile:

Due to copyright issues I can’t share the finished program with the intended images. But when I’m done I’ll try to make a “dummy version” where all the code is intact and only the images have been swapped, in case it’s of any help to someone who stumbles upon this thread and wants to try something similar.

Edit1: Re: “… so you can stop drawing them at the end of the trial” - I added this to the code snippet to make sure that the crosses aren’t drawn afterwards, I’m guessing you meant something like it.

# End Routine
for across in crosses:
        across.setAutoDraw(False)

Maybe there’s a more elegant way to do it, I basically just copied the method used by PsychoPy itself for stopping objects from drawing themselves.

Edit2: While looking at the documentation for ShapeStim (https://www.psychopy.org/api/visual/shapestim.html#psychopy.visual.ShapeStim) I noticed that it says the “depth” parameter is deprecated, because “Depth is now controlled simply by drawing order.”… But when using code snippets in the Builder, just reordering the stimuli doesn’t seem to work (since the code snippet isn’t arranged in the actual code like other stimuli as far as I understand), and the crosses end up being “underneath” the other stimuli if the depth parameter isn’t used. I wonder what the best-practice method for solving this would be?

1 Like

Perhaps don’t use autoDraw but instead explicitly draw the stimuli yourself on every frame, e.g.:

# this first step may or may not be necessary, but if it is, you'll probably need to set 
# the corresponding Builder component to not have a time of onset:
background_image.draw()

for cross in crosses:
    cross.draw()
1 Like

@Michael This is a late reply, but thanks for the tip! At the moment I’m using the “uglier” method of specifying the ‘depth’ method. It’s been so long now that I can’t remember why I ended up doing so, but I’ll have a new go at implementing your solution once I get everything else working as it should.

Other tasks have gotten in the way so developing the experiment is something I’ve had to do little by little on the side, but now I have some more time to focus on getting it to work properly. The functionality itself is basically solved after I built on @dvbridges tips . Now however I’ve run into performance issues. When running the experiment as is planned, with ca. 60 seconds/image for 5 images, my computer (a 2015 MacBook Pro, so not great but not terrible either) heats up a lot. I tried running a copy of the experiment with the “clicks producing crosses” code snippets removed, but it seems like it’s just the rendition of images itself that’s causing the performance issues. The images are about 700KB in size. I can’t compress them too much because small details will be important in the experiment. It’s likely that they could be a bit smaller than that though, since I know nearly nothing about image compression and doing it properly would probably make a difference. So they might be unnecessarily large, but at the same time I’m thinking there should be a solution for using somewhat bigger images in PsychoPy. Is there a way to make PsychoPy (/pyglet) render a “static” image so that it doesn’t need to be redrawn for every win.flip(), or at least a way to cache the image in some way that would reduce the computational load? I can’t reduce the frequency of screen refreshes itself because I still want the crosses to be drawn on the polygon as quickly as possible.

I’ve uploaded a version of the experiment where I’ve replaced all the copyrighted images with a placeholder image. It has all the instructions and functionality of the actual experiment, so it might be helpful for others. I would be very grateful if someone could try running the experiment and see if it produces the same performance/heat issues. Github repo link: https://github.com/AnonZebra/psychopy_find_targets

File size has little to do with the resulting image size. e.g. take two images of 1680 × 1050 pixels. A uniform black rectangle would require a tiny .png file to represent it, while completely random noise of colour pixels would hardly compress at all, and take several megabytes on disk. However, each would decompress to the same 1680 × 1050 × colour channels = nearly six megabytes. i.e. the image sent to your graphics card is uncompressed, so the size of the file used to save it on disk is not relevant.

Image size is also not in itself going to be responsible for performance issues. If the image is unchanging for 60 seconds, that requires almost no effort for the graphics card to keep redrawing at say 60 Hz. It is immensely efficient at that sort of thing.

1 Like

Thank you! That’s true about the original image file size making no difference, I had read somewhere else about that but forgotten. So that’s not the issue. I tried running PsychoPy in windowed mode (setting fullscreen=False for the window instance), and looked at CPU usage. It turns out that:

  1. My issue is primarily Mac-specific and not related to only PsychoPy, but Python in general. Brief description in case anyone else gets this problem: There’s a process called “tccd” that’s supposed to be handling matters related to the Contacts list, but due to some bug with Mojave/Catalina versions of Mac OSX, it hogs a lot of CPU/memory for certain apps that aren’t even in the list of apps that could get access to contacts. Python seems to be one of them. I have found no solution for this unfortunately. See e. g. https://apple.stackexchange.com/questions/352454/system-slowness-how-to-disable-or-short-circuit-tccd-in-mojave-slows-app-su

  2. When I run the experiment on a fairly old Windows computer it seems to work fine. Only thing is that the CPU usage jumps up a fair bit when I draw a bunch of crosses (from about 7% to 15-20%) on the screen. Everything still works, it just seems odd that the crosses/mouse input should lead to such a spike in computational processing.

Issue 1 is incredibly annoying but I realize it’s not something I can ask for help with here (should anyone happen to know anything please let me know though).

I added a click “cooldown period” of .2 seconds to the experiment, including the version in the github repo I linked earlier. It should be safe for anyone wanting to do something similar to copy and build on the repo, since the processor strain issue isn’t caused by the experiment setup itself. As it is right now, I did this:

#Begin experiment
def makeCross(pos):
    cross = visual.ShapeStim(win, pos=pos, depth=-2, vertices='cross', size=(5,5), lineColor=[-0.8, -0.8, 1], fillColor=[-0.8, -0.8, 1])
    cross.setAutoDraw(True)
    return cross
#Begin routine
crosses = []
last_press_time = 0
#Each frame
time_now = core.getTime()
if practice_mouse.isPressedIn(practice_poly) and (time_now - last_press_time) > 0.2:
    crosses.append(makeCross(practice_mouse.getPos()))
    last_press_time = time_now

I couldn’t find a better solution in order to prevent a swathe of crosses being formed if a participant keeps the mouse button pressed while moving around. It would be even better if I could change the script to only respond to actual clicks/“new mouse presses”, since if the participant keeps the button pressed for >.2s two crosses will be created anyhow as it is now. I couldn’t find how to do that in the docs though.

A part of the reason for your spike in CPUI usage could be that you are re-creating stimuli on every screen refresh. This is inefficient. Creating stimuli from scratch (as opposed to just changing their attributes, like position, orientation, etc) is very time-expensive. So you should just create this cross stimulus once and then draw it in as many positions as required. e.g. something like:

if etc:
    position_list.append(practice_mouse.getPos())

# maintain a list of positions:
for position in position_list:
    cross_stimulus.pos = position
    cross_stimulus.draw() # draw one stimulus many times

You need to keep track of both when the mouse is pressed and then when it is released. e.g. set flags at the beginning of a routine like: mouse_has_been_pressed = False and mouse_has_been_released = True.

When you detect that the mouse has been pressed, only proceed to draw a cross if also mouse_has_been_released. In that code, then set mouse_has_been_released = False. If the mouse button is not pressed, restore mouse_has_been_released = True

Thank you! I modified the snippet like so:

if practice_mouse.isPressedIn(practice_poly) and mouse_has_been_released:
    cross_pos_ls.append(practice_mouse.getPos())
    mouse_has_been_released = False

elif not (practice_mouse.getPressed()[0]==1 or mouse_has_been_released):
    mouse_has_been_released = True

practice_context.draw()
# You have to manually disable/comment out the AutoDraw
# code in the generated script, these are the lines:
# practice_context.setAutoDraw(True)
# practice_context.setAutoDraw(False)

for pos in cross_pos_ls:
    cross.pos = pos
    cross.draw(win)

Like it says in the comments, the AutoDraw code lines generated by PsychoPy Builder have to be disabled manually. The reason is that everything that’s set to AutoDraw is drawn at the very end, so the picture would end up blocking the view of all the crosses. It’s kind of a dirty way to solve this problem though, maybe I’ll try rewriting the code in due time using something like what you mentioned in this thread Order of autodraw stimuli

As far as I can tell, the crosses don’t seem to be having as much of an effect on CPU usage as before now that only one instance is used, like you suggested might be the case.

Hello in 2020. Did you try your experiment online ? I tried using the custom makeCross function and .isPressedin() in an online task. It did not work. It worked in builder.
Thanks

Hi,

No, I’m afraid I didn’t try out the experiment online. The few times I’ve tried to convert other experiments to javascript/for online use, it’s never worked as expected. Even the example online experiments from the PsychoPy team haven’t worked that well (of course, my browser may be at fault here), and since I don’t really have a need for it I’ve basically given up on playing around with PsychoPy’s online functionality. I hope things work out better for you though.

1 Like