Any VR application?

Blog
1

OS: Ubuntu 16.04
Psychopy version: 1.90

Anyone has tried Psychopy in a VR or AR environment? Where should I start? Any tutorial available?

Thank you

1 Like
2

I actually have developed modules for using the Oculus Rift CV1 with PsychoPy, allowing for full 3D scenes to be rendered with tracking. I haven’t yet made them publicly available since there is some preliminary work required to incorporate such a massive code base into PsychoPy without breaking everything.

Stay tuned.

4 Likes
3

Great! Looking forward to, and please keep me (and the psychopy community) informed !!

ty

4

That’s awesome! I’ve been trying to get any stimulus presentation set up with an Oculus CV1 with OpenHMD, with little luck. Looking forward to a public release – any ETA? Would love to talk to you a bit about what you’ve accomplished there!

5

Hello,

Interesting you brought this up now since I’ve been working on getting this out ASAP. I updated the bindings to support the most recent version of the Oculus SDK, but I need to put a bit more work into getting the PsychoPy interface to reflect those changes.

At the very least, the extension library for working with the Rift will be released soon as a separate package. If you have any questions, you can private message me them.

2 Likes
6

That’s awesome: PM’ed!

Looking forward to a release!

7

Great!! Please keep me (and the Psychopy community) informed. I started using Psychopy just a year ago. I love it. Before that, I used E-prime and customized C# and VB.net. Psychopy is the best. I earnestly hope the Psychopy framework grows and becomes the standard for behavioral and neuroscience studies.

  • takashi
1 Like
8

Hello,

I’ve publicly released the source code for a library used to interface with the Oculus Rift (CV1 and DK2) called PsychXR (https://github.com/mdcutone/psychxr). The library is the basis for HMD support being added into PsychoPy. PsychXR just wraps HMD APIs does not provide rendering support, you need to use Pyglet or similar to draw anything. Eventually other APIs may be supported through that library, however there are no plans as of yet (OpenHMD for instance).

I would wait for the PsychoPy support which greatly simplifies using the Rift in your experiments, including 3D stimuli classes and examples. However, if you are writing applications from scratch and can handle OpenGL, you can use PsychXR directly. I’ll be releasing pre-compiled binaries of PsychXR once PsychoPy support rolls out since their development is closely related.

9 Likes
9

Just a follow-up.

PsychXR is now available on PyPI. Only 64-bit Python 3.6 is supported.

1 Like
10

Thank you! takashi

11

Hello,

Just to let everyone know that PsychoPy 3 has Rift support along with some demos. You need PsychXR installed to use it. I’m looking into a pure-Python replacement for PsychXR to eventually remove that as a requirement, but you need to use it for now. There is also some decent support already in PsychoPy 3 for loading *.OBJ files (with textures and normals) for use in VR scenes. I spent a bit of time optimizing 3D model generation to reduce as much overhead as possible. Right now models are rendered using the old ‘red book’ OpenGL spec (1.1 if I recall) for compatibility purposes, however, I use modern VAOs and indexed draw buffers which provide large performance gains for more demanding applications. I’ll write up a short guide to loading 3D models into PsychoPy for use with the Rift when I have the chance.

5 Likes
12

Update, I just pushed the new version of the Rift class which supports the much improved PsychXR 0.2+. It should ship with PsychoPy soon. Note that VR support is limited to Windows 64-bit platforms and the Oculus CV1 and S headsets for now.

Here are some notable features now included:

  • You are now able to easily use arbitrary head pose data to compute eye locations.
  • Removes all of the math functions from the Oculus library in favor of the new LibOVRPose. This greatly simplifies rigid body transformations in VR scenes by having a single class deal with it (much like the Pose class in Unity). If you want to do some math that is not available through LibOVRPose's methods, nearly all of it is in mathtools library now.
  • All math types such as vectors, matrices, and quaternions are now numpy arrays. So you can use them more readily with your code. The LibOVRPose object will have a common interface with the incoming RigidBodyPose class for the 3D stim class. This allows you to directly use pose data from the Rift to transform 3D stimuli.
  • Added the ability to make movies of what your participant is seeing.
  • Included support for PsychXR’s visibility culling feature, this allows you to selectively render objects only if they appear in the FOV of a given eye. This requires using the new LibOVRBounds object and attaching it to a LibOVRPose representing your scene object.
  • You can now track additional touch controllers in a scene using the getDevicePoses method.
  • Support for additional mirror texture modes.
  • getTrackingState and getDevicePoses gives information about dynamics such as linear and angular velocity and acceleration.
  • Support for haptics has been added. This allows you to control touch controller vibration. The LibOVRHapticsBuffer object can be used to map vibration samples stored in numpy arrays to the haptics engine for playback.
  • Performance statistics from the LibOVR runtime is now accessible. It can report instances were frames were dropped by the application.

Recent changes to PsychoPy should allow me to finally push out the 3D stimulus class, which supports loading *.obj files with textures, materials, and basic lighting. There are also other classes for shapes like boxes, planes and spheres which can be textured and shaded. I’m hoping to get that out soon.

2 Likes
13

Dear Matthew, I am stuck trying to get a simple 2D experiment to run on an HTC Vive.
I know the oroblem here is my low knowledge level, but I believe you cold take me out of the swamp in just one minute.
I would really appreciate if you colud point me in the right direction, details are at this topic

Thank you!

Francesco

14

Hello Francesco,

Unfortunately, all non-Oculus HMDs are currently unsupported by PsychXR and therefore PsychoPy. Support can be added someday to PsychXR or through some other means, but that would involve a lot of work at this point.

More on PsychXR: http://psychxr.org/

  • mdc