Presenting multiple stimuli incrementally

Coding
1

Hi everyone,

I hope this isn’t answered elsewhere and I missed it, but I’m having doing trouble doing something I thought would be simple. I’m using PsychoPy version 1.90.3 for Python 2.7, under Windows 7.

In my experiment, I want participants to make a categorization judgement, and get feedback about their accuracy. I thought I would do this by drawing the stimulus to be categorized to the screen, and then after the judgement was made, add some text telling them correct vs. incorrect. This simple example below describes what I want to happen:

from psychopy import core, visual

win = visual.Window([400, 300])

# This text should appear along on the screen for 2 seconds
message1 = visual.TextStim(win, text="Greetings.")
message1.draw()
win.flip()
core.wait(2.0)

# This text should then be *added* to the screen along with the "Greetings" text
message2 = visual.TextStim(win, text="My name is Hal 9000", pos=(0,-.25))
message2.draw()
win.flip()
core.wait(2.0)

win.close()
core.quit()

But instead, what happens is the second message appears by itself. In Psychtoolbox-land, I would add the second message to the screen by asking it to not clear the back buffer when “flipping” the stimuli to the screen. But in PsychoPy-world, I can only choose not the clear the front buffer when flipping.

Is there any way I can add to what is on the screen currently? I’d rather like to avoid redrawing the entire contents of the screen after each flip. If I’m forced to overwrite the back buffer, is there any way I can copy the front buffer to the back buffer without “flipping” the two?

EDIT: Am I going to have to abuse auto-drawing here?

Thanks in advance for the guidance.

2

Hello,

The ‘flip’ function has an optional ‘clearBuffers’ argument. Set it to False to keep the text up across flips.

3

Hi @mdc, thanks for your quick reply. That’s exactly what I thought to do, but it doesn’t produce the result I’m looking for.

It seems that win.flip(clearBuffer=False) makes it so the front buffer isn’t cleared when the front and back buffers are flipped. Look at the result of this code:

from psychopy import core, visual

win = visual.Window([400, 300])

# This text should appear along on the screen for 2 seconds
message1 = visual.TextStim(win, text="Greetings.")
message1.draw()
win.flip()
core.wait(2.0)

# This text should then be *added* to the screen
message2 = visual.TextStim(win, text="My name is Hal 9000", pos=(0,-.25))
message2.draw()
win.flip(clearBuffer=False)
core.wait(2.0)

# This will make the "Greetings" text show up again, because the previous 
# win.flip(clearBuffer=False) put it on the back buffer! 
win.flip()
core.wait(2.0)

win.close()
core.quit()

It seems like the clearBuffers=False argument would allow me to present the previous screen again later, but doesn’t help me add things to the screen incrementally.

4

The general PsychoPy drawing model is that the screen is erased on every flip, so you need to draw everything you want displayed prior to each flip. So in the first case, draw just the first text, then flip the window. In the second case, simply draw both stimuli before flipping.

message1.draw()
win.flip()
core.wait(2.0)

message1.draw()
message2.draw()
win.flip()
core.wait(2.0)

And ideally pre-create your text stimuli before you get to the drawing stage. They are time-expensive to initialise (and to a lesser extent, to update the text content).

5

OK, I understand. I did feel like I was going against the grain somehow, good to know I’m not crazy.

One one hand, this model makes sense - lots of PsychoPy experiments are created with the Builder interface, which inherently keeps track of what happens on each frame, and can easily generate corresponding code (i.e., .draw() commands).

On the other hand, when you’re not using the Builder GUI, the “screen is always cleared, you must re-draw everything always” model can become cumbersome to work with. You have to track everything you want on the screen, and potentially carry these objects with you all around the code base.

Imagine a visual search task where you want the participant to find a bunch of items in the display, and click on them to mark them as “found” with a red circle. I’d need to keep a list of all the circle stimuli created so far, and redraw them all each time a another item is clicked on. If I could just say “don’t clear the back buffer”, this task gets a simpler and more efficient.

Or, consider a task where I want a participant to rate some item on several scales, and have each scale update visually in some manner to reflect the participants choice. This has the same “keep copies of everything so far” complexity drawback of the visual search example, but here I might also want to modularize my code by having a function that is responsible for drawing and processing responses on each rating scale. But this means I now have to pass around the objects and state for each rating scale to every individual function, for the purposes of redrawing. This completely defeats the purpose of having a modular function. Yes, there is *args, but it’s still not conceptually right.

I think having the ability to “autodraw” things mitigates the issue somewhat, but you still have track what you want autodrawn and what you don’t so you can turn it on or off as needed, which isn’t necessarily a trivial problem.

I hope I don’t sound confrontational, or come off as an arrogant know-it-all, I just think there are reasonable situations where having the ability to say “don’t clear the back buffer please” would make tasks simpler, and wanted to share this perspective.

If I was a stubborn person and didn’t want to .draw() all the things, are there any potential downsides of using win._getFrame(buffer='back') to save a copy of my precious back buffer before flipping, and pass that around to be “redrawn”?

6

Maybe that task does, but it’s not a general solution. The back buffer is just a 2D matrix of memory. There is no concept of drawing order. So anything you want left in the buffer must always be background material and anything subsequently gets drawn on top. Nothing can easily be “undrawn”. That is a drawing model that was abandoned a long time ago in most corners of software design.

A good example of that would be when you mention using rating scales. If the back buffer isn’t erased, then using a rating scale interactively becomes impossible: the pointer will leave ghost images behind every time it is moved.

This drawing model strangely doesn’t seem to be an issue for most people. Use autodraw if it’s easier for you, and keep related stimuli in lists that can easily be iterated over for drawing or setting/unsetting the autodraw property, or use the element array stimulus if it works for you, to handle drawing lots of related stimuli quickly in one fell swoop.

7

It seems that win.flip(clearBuffer=False) makes it so the front buffer isn’t cleared when the front and back buffers are flipped.

Right, that’s what you would expect now that I think about it. The buffers are being swapped, so the front buffer’s content is not showing up in the back buffer unless flipped again.

In Psychtoolbox-land, I would add the second message to the screen by asking it to not clear the back buffer when “flipping” the stimuli to the screen.

PTBs behaviour here is actually the side-effect of the method it uses to synchronize drawing (your program) with the monitor’s vertical retrace. Here is a snippet from PTBs source code describing this process…

From SCREENWaitBlanking.c :

Other methods unsupported. We use the doublebuffer swap method of waiting for retrace.
Working principle: On each frame, we first copy the content of the (user visible) frontbuffer into the backbuffer.
Then we ask the OS to perform a front-backbuffer swap on next vertical retrace and go to sleep via glFinish() et al.
until the OS signals swap-completion. This way PTB’s/Matlabs execution will stall until VBL, when swap happens and
we get woken up. We repeat this procedure ‘waitFrames’ times, then we take a high precision timestamp and exit the
Waitblanking loop. As backbuffer and frontbuffer are identical (due to the copy) at swap time, the visual display
won’t change at all for the subject.
This method should work reliably, but it has one drawback: A random wakeup delay (scheduling jitter) is added after
retrace has been entered, so Waitblanking returns only after the beam has left retrace state on older hardware.
This means a bit less time (1 ms?) for stimulus drawing on Windows than on OS-X where Waitblanking returns faster.

Note the above describes one of PTB’s many wait blanking methods, however this method is likely being used.

In general, swapping buffers is non-blocking, meaning that your program can keep issuing draw commands asynchronously, whether or not the buffer’s contents have been displayed (Synchronization - OpenGL Wiki). This leads to graphical glitches (e.g. tearing) and poor timing. PsychoPy and PTB both issue glFinish commands which lock up your program until some dummy draw commands are completed, which won’t happen until the back-buffer is freed by the GPU driver after retrace. PTB’s dummy commands involve copying the front buffer to the back, which results in the back-buffer mirroring the front once the back-buffer is released. So if the buffer isn’t cleared, the copied image will remain. PsychoPy issues different commands for glFinish to block that doesn’t draw anything visible to the back-buffer.

8

100% agreed. Still feels like a nice option to have.

Good point, I didn’t consider the mouse cursor. It’s abstracted away from your experiment code, so it’s easy to forget about.

I think I’ll be able to handle it now that I know whats possible/not possible, and what the best practices here are. Thanks for the pointer about the element array stimulus, I’ll look into that. Anyway, thanks for letting me know I’m doing_it_wrong

9

I disagree that you’d expect that, but it’s sensible enough. I don’t think the docs make it clear (the parenthetical about “present the previous screen” hints at it), and from previous PTB experience I assumed it was talking about the back buffer.

Interesting, it’s kind of an implementation detail then, thanks for the insight. From your quote, I’m missing when the things you have drawn and want displayed are put onto the front buffer (the quote starts by saying that you take the front buffer and copy it to the back buffer), but that’s probably a different discussion altogether.

10

The quote comes from a comment in PTB’s code describing what it does at the end of a flip for vertical trace synchronization. This is done automatically.

The point here is that you are expecting to draw over the image of your previous frame’s buffer. That only works for PTB because it copies pixel data from the front buffer to the back buffer prior to releasing it to you after flipping. This means that you have a copy of your front buffer on your back buffer at the start of a frame. If you don’t clear the back buffer, anything you draw appears on top of what was drawn the previous frame.

11

Ah, so this quote is talking about syncing to the monitors refresh cycle, not all presentation more generally?

You made me curious so I started poking around the PTB repo, and I think the SCREENwaitblanking stuff is a legacy mode that’s not used anymore (I don’t know C other than to recognize it, so I might be wrong), and in the newer stuff it looks like they are much more explicit about restoring the backbuffer after flips now. If you’re curious, check out the PsychPostFlipOperations function, and the PsychFlipWindowBuffers function that calls it.

12

Thanks for the links. I haven’t used PTB for some time, but I would expect they kept the behavior consistent in the new pipeline. In sum, PsychoPy’s drawing model is different than PTB’s and Michael suggestion would give you the results you ask for.