I am new to Psychopy and I would like to know whether there is a function that allows drawing dots with 3 coordinates X, Y and Z? I found the function DotStim but it only takes X and Y coordinates.
Thanks in advance for your help!
1 Like
Right now you need to use OpenGL calls to generate 3D stimuli. PsychoPy has some tools to help setup the projection in psychopy.tools.viewtools. There is also psychopy.tools.mathtools for creating model/view matrices.
import ctypes
import pyglet.gl as GL
import psychopy.tools.viewtools as vt
import psychopy.tools.mathtools as mt
# setup projection matrix, get `scrWidth`, `scrAspect`, and `scrDist` from monitor settings
scrDist = 0.50 # 50cm
scrWidth = 0.53 # 53cm
scrAspect = 1.778
frustum = vt.computeFrustum(scrWidth, scrAspect, scrDist)
P = vt.perspectiveProjectionMatrix(*frustum)
# transformation for points (model/view matrix)
MV = mt.translationMatrix((0.0, 0.0 , -scrDist)) # X, Y, Z
## --- inside the rendering loop ---
# draw points
GL.glMatrixMode(GL.GL_PROJECTION)
GL.glLoadTransposeMatrixf(P.ctypes.data_as(ctypes.POINTER(ctypes.c_float)))
GL.glMatrixMode(GL.GL_MODELVIEW)
GL.glPushMatrix()
GL.glLoadTransposeMatrixf(MV.ctypes.data_as(ctypes.POINTER(ctypes.c_float)))
# draw points or triangles here
GL.glBegin(GL.GL_POINTS)
GL.glColor4f(1.0, 1.0, 1.0, 1.0)
GL.glVertex3f(0.0, 0.0, 0.0) # position
GL.glEnd()
GL.glPopMatrix()
After calling the above commands, you may need to restore the original projection the window class was using if you chose to draw anything else in 2D.
win.resetEyeTransform()
The window class also has some attributes to simplify setting up these transformations, but the support is experimental at this time. Furthermore, PsychoPy is going to be getting comprehensive, high-performance (and I mean it!), 3D stimuli support in the near future to compliment the VR interface. You’ll soon be able to handle drawing thousands of points in a single command without dropping a frame.
EDITED: Fixed the transformation to be along the Z axis. Fixed some other issues too.
4 Likes
After some consideration, looks like you need to use the new window projection features to get the results you want. Try this instead of what was shown before.
scrDist = 0.50 # 50cm
scrWidth = 0.53 # 53cm
scrAspect = 1.0
frustum = vt.computeFrustum(scrWidth, scrAspect, scrDist, nearClip=0.1, farClip=1000.0)
P = vt.perspectiveProjectionMatrix(*frustum)
# transformation for points (model/view matrix)
MV = mt.translationMatrix((0.0, 0.0, -scrDist)) # X, Y, Z
win.projectionMatrix = P
win.viewMatrix = MV
# --- render loop ---
win.applyEyeTransform()
# draw 3D stuff here
GL.glBegin(GL.GL_POINTS)
GL.glColor4f(1.0, 1.0, 1.0, 1.0)
GL.glVertex3f(0.0, 0.0, 0.0) # position
GL.glEnd()
win.resetEyeTransform()
# draw 2D psychopy stimuli here ...
This greatly simplifies working with 3D stimuli (which is what I intended). Here’s the result I get when rendering a 3D model…
1 Like
Thanks very much for your answers and code samples!
I tried to use your second code sample and added win and monitor parameters as well as the 2D Psychopy stimulus under “# draw 2D Psychopy stimuli here …”. However, the 3d rendering didn’t change my stimulus. It is still a 2D one.
from psychopy import event, monitors
import psychopy.visual,psychopy.event
import random
import numpy
from dataclasses import dataclass, field
from typing import List, Tuple
from ipynb.fs.full.secs2frames import secs2frames
import ctypes
import pyglet.gl as GL
import psychopy.tools.viewtools as vt
import psychopy.tools.mathtools as mt
#Monitor settings
widthPix = 1920 # screen width in px
heightPix = 1080 # screen height in px
monitorwidth = 53.1 # monitor width in cm
viewdist = 60. # viewing distance in cm
monitorname = 'BOE CQ LCD'
scrn = 0 # 0 to use main screen, 1 to use external screen
mon = monitors.Monitor(monitorname, width=monitorwidth, distance=viewdist)
mon.setSizePix((widthPix, heightPix))
scrDist = 0.50 # 50cm
scrWidth = 0.53 # 53cm
scrAspect = 1.0
# Create a window
win = psychopy.visual.Window(
monitor=mon,
size=(widthPix,heightPix),
color='Gray',
colorSpace='rgb',
units='deg',
screen=scrn,
allowGUI=True,
fullscr=True)
#Frustum
frustum = vt.computeFrustum(scrWidth, scrAspect, scrDist, nearClip=0.1, farClip=1000.0)
P = vt.perspectiveProjectionMatrix(*frustum)
# Transformation for points (model/view matrix)
MV = mt.translationMatrix((0.0, 0.0, -scrDist)) # X, Y, Z
win.projectionMatrix = P
win.viewMatrix = MV
# --- render loop ---
win.applyEyeTransform()
# draw 3D stuff here
GL.glBegin(GL.GL_POINTS)
GL.glColor4f(1.0, 1.0, 1.0, 1.0)
GL.glVertex3f(0.0, 0.0, 0.0) # position
GL.glEnd()
win.resetEyeTransform()
# draw 2D psychopy stimuli here ...
# Create a class of dots attributes
@dataclass
class dots_attributes:
n: int = 200
xy: List = field(default_factory=list)
color: Tuple = field(default_factory=lambda:(255,255,255))
size: int = 10
center: List = field(default_factory=lambda:[0,0,0])
#theta=numpy.random.rand(nDots)*360
#dotsRadius=(numpy.random.rand(nDots)**0.5)*200
# Create a class of animation attributes
class animation_attributes:
framerate: float = win.getActualFrameRate(nIdentical=10, nMaxFrames=100, nWarmUpFrames=10, threshold=1)
duration: int = 10
screen_size: List = field(default_factory=lambda:[400,400])
dots = dots_attributes()
anim = animation_attributes()
# Loop through the number of dots to create x and y coordinates of size dots.n
for dot in range(dots.n):
dot_x = random.uniform(-widthPix/2, widthPix/2)
dot_y = random.uniform(-heightPix/2, heightPix/2)
dots.xy.append([dot_x, dot_y])
#Draw the stim on the window
dot_stim = psychopy.visual.ElementArrayStim(
win=win,
units="pix",
nElements=dots.n,
elementTex=None,
elementMask="circle",
xys=dots.xy,
sizes=10
#colors = dots.color
)
Nframes = secs2frames(anim.framerate,anim.duration)
print(Nframes)
dot_stim.draw()
win.flip()
psychopy.event.waitKeys()
win.close()
Perhaps I’m not understanding the issue. Are you trying to draw objects in a 3D coordinate system, or do you want to render things in stereo 3D?
Points don’t have many depth cues, so it might not appear to be working since the point you are rendering would be in the dead centre of the screen.
- mdc
I am trying to draw dots in a 3D coordinate system; each dot will have x,y and z coordinates. I also want to draw them from the perspective of an observer. Therefore, I need to make a perspective projection as well. My main goal is to code a radial optic flow – I just uploaded a GIF of an optic flow animation. The dots move from a focus of expansion (vanishing point) towards the observer. I am not doing a stereoscopic one yet. The dots that are far away are smaller and slower, and the dots that are closer to the observer are bigger and faster.
Interesting, I was working on a demo for the new window features that looks like your GIF. I’ll try to get it working and post the code. I’ll get back to you later today. Soon…
FYI, I’m pushing new OpenGL features to PsychoPy that can update and render >100k points per frame (nearly 1 million on my GPU). However, right now we need to use low level GL calls in our rendering loop that can handle a few 100.
Awesome, I look forward to seeing the new features and trying them out!
Try something like this out …
scrDist = 0.50 # 50cm
scrWidth = 0.53 # 53cm
scrAspect = 1.0
# Create a window
win = psychopy.visual.Window(
monitor=mon,
size=(800, 800),
color='Black',
colorSpace='rgb',
units='deg',
screen=scrn,
allowGUI=True,
fullscr=False)
#Frustum
frustum = vt.computeFrustum(scrWidth, scrAspect, scrDist, nearClip=0.1, farClip=1000.0)
P = vt.perspectiveProjectionMatrix(*frustum)
# Transformation for points (model/view matrix)
MV = mt.translationMatrix((0.0, 0.0, -scrDist)) # X, Y, Z
win.projectionMatrix = P
win.viewMatrix = MV
# create array of random points
nPoints = 10000
pos = np.zeros((nPoints, 3), dtype=np.float32)
# random X, Y
pos[:, :2] = np.random.uniform(-500, 500, (nPoints, 2))
# random Z to far clipping plane, -1000.0 is -farClip
pos[:, 2] = np.random.uniform(0.0, -1000.0, (nPoints,))
while 1:
# --- render loop ---
win.applyEyeTransform()
# draw 3D stuff here
GL.glColor3f(1.0, 1.0, 1.0)
GL.glBegin(GL.GL_POINTS)
for i in range(nPoints): # go over our array and draw the points using the coordinates
# color can be varied with distance if you like
GL.glVertex3f(*pos[i, :]) # position
GL.glEnd()
win.flip()
# transform points -Z direction, in OpenGL -Z coordinates are forward
pos[:, 2] += 1.0 # distance to move the dots per frame towards the viewer
# if a point its behind us, return to initial -Z position
pos[:, 2] = np.where(pos[:, 2] > 0.0, -1000.0, pos[:, 2])
# check events to break out of the loop!
Elementstim cannot do 3D rendering at this point, it constrains everything to a plane in depth. So for the time being, you need to do something like above. Be sure to call resetEyeTransform before drawing 2D psychopy stimuli after the dots are drawn.
2 Likes
Awesome! thanks! I tried it out and it worked.
I played with the code and some parameters (dots size and positions). I can see that dots speed increases when it get closer to the viewer. However, their size doesn’t increase when they move towards the viewer.In fact, the size remains the same. I expected the distance to be a function of both the speed and the size. Do you think it’s something I should do manually ?
#Monitor settings
widthPix = 1920 # screen width in px
heightPix = 1080 # screen height in px
monitorwidth = 53.1 # monitor width in cm
viewdist = 60. # viewing distance in cm
monitorname = 'BOE CQ LCD'
scrn = 0 # 0 to use main screen, 1 to use external screen
mon = monitors.Monitor(monitorname, width=monitorwidth, distance=viewdist)
mon.setSizePix((widthPix, heightPix))
scrDist = 0.50 # 50cm
scrWidth = 0.53 # 53cm
scrAspect = 1.0
# Create a window
win = psychopy.visual.Window(
monitor=mon,
size=(1000, 800),
color='Black',
colorSpace='rgb',
units='deg',
screen=scrn,
allowGUI=True,
fullscr=False)
#Frustum
frustum = vt.computeFrustum(scrWidth, scrAspect, scrDist, nearClip=0.1, farClip=10000.0)
P = vt.perspectiveProjectionMatrix(*frustum)
# Transformation for points (model/view matrix)
MV = mt.translationMatrix((0.0, 0.0, -scrDist)) # X, Y, Z
win.projectionMatrix = P
win.viewMatrix = MV
# create array of random points
nPoints = 900
pos = np.zeros((nPoints, 3), dtype=np.float32)
# random X, Y
pos[:, :2] = np.random.uniform(-500, 500, (nPoints, 2))
# random Z to far clipping plane, -1000.0 is -farClip
pos[:, 2] = np.random.uniform(0.0, -1000.0, (nPoints,))
while 1:
# --- render loop ---
win.applyEyeTransform()
# draw 3D stuff here
GL.glColor3f(1.0, 1.0, 1.0)
GL.glPointSize(5.0);
GL.glBegin(GL.GL_POINTS)
for i in range(nPoints): # go over our array and draw the points using the coordinates
# color can be varied with distance if you like
GL.glVertex3f(*pos[i, :]) # position
GL.glEnd()
win.flip()
# transform points -Z direction, in OpenGL -Z coordinates are forward
pos[:, 2] += 5.0 # distance to move the dots per frame towards the viewer
# if a point its behind us, return to initial -Z position
pos[:, 2] = np.where(pos[:, 2] > 0.0, -1000.0, pos[:, 2])
# check events to break out of the loop!
if len(event.getKeys())>0:
break
event.clearEvents()
win.close()
Not sure how to vary size with distance. There is a command glPointSize, but if can recall, it doesn’t permit being set within the glBegin/glEnd statements. If you set it every before every call to glVertex2f, it will slow down your program quite a bit. I know how to draw points with fragment shaders, so I’ll look into it.
Thanks! In fact, I added GL.glPointSize(5.0) in the above code, before GL.glBegin(GL.GL_POINTS), to play with the size of the dots, and increased the distance to see the variation of speed and size. I assume that just as the increase of speed, the increase of size would also vary with the distance with a projection perspective.
Ok I found out why there is no change of size. When we draw a point with GL_POINTS, it’s not a 3D object with a size in world-space: its size is a fixed number of pixels. Because of that, the size of the point is independent of the distance from the camera. If I want to draw dots whose size depends on distance, I need to use another GL method to draw them; I was recommended to draw them with fragment shaders (that’s probably why you also mentioned this method).
@mcd I need to change the field of view values, and I would like to know how would I calculate it with the variables and functions in your script. For instance, if I want an angle of 40°, how can I implement it with the scrAspect and the perspective projection function?
Thanks very much for your help!!
Hi Kathia,
I’m a bit busy this week, sorry for the late reply.
If you want a specific projection for a given visual angle, you’ll need to compute a projection matrix as shown here:
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/gluPerspective.xml
The functions supplied with PsychoPy were intended to work with the monitor calibration data, so the FOV is computed from that. The provided link will show how to compute a projection from an FOV, you just need to create the martix and set it as your projection matrix.
-mdc
Hi mcd,
Thank you for your answer, and the link!
Does it mean that I have to give up on the function P = vt.perspectiveProjectionMatrix(*frustum) and replace it with gluPerspetive ?
Many thanks again for your help!
In this case you’ll need to implement gluPerspective yourself to work with the current system, something like this may work (haven’t tried it out):
def glu_perspective(fovy, aspect, near, far):
f = np.tan(fovy / 360. * np.pi) * near
m00 = f / aspect
m11 = f
m22 = (far + near) / (near - far)
m23 = (2 * far * near) / (near - far)
return np.array([
[m00, 0, 0, 0],
[ 0, m11, 0, 0],
[ 0, 0, m22, m23],
[ 0, 0, -1, 0]],
dtype=np.float32)
Then you set this as the window’s view matrix.
1 Like
Adding some functions to PsychoPy to help with this stuff, including the gluPerspective stand in. There is also a function that will allow you to determine the size of the near and far clipping planes, so you can randomly generate points that all fit within the frustum volume.
@mdc Awesome, what function is it? Should I update Psychopy to get this functionality? Thanks very much for your precious help!
Waiting in staging and not released yet unfortunately. Description of the new functions are here: https://github.com/psychopy/psychopy/pull/2870
2 Likes