How do you track pupil size live on iohub?

Coding
1

Hi all!

I’m trying to code up an eye tracking script that is able to execute a certain command when a blink has been detected.
The way I’m doing it right now (as shown below) is that when no gaze position is detected, execute the certain command.

This is not exactly ideal since saccades off the screen would also be considered as a ‘blink’ in this case. There wasn’t any explicit commands listed in the iohub guide that I thought could be use to measure pupil size live. Does anyone have an advice for this?

blinked = 0

for disptime in range(displaytime):

reopened = False #track if eyes reopened

# Get the latest gaze position in display coord space.   
gpos=tracker.getLastGazePosition()
if isinstance(gpos,(tuple,list)):
    # If we have a gaze position from the tracker, draw the 
    # background image and then the gaze_cursor.
    gaze_dot.setPos(gpos)
    gaze_dot.draw()
else:
    blinked = 1
    print 'Blink occurred just before/at:', stimlist[counter]
    while reopened == False:
        gpos=tracker.getLastGazePosition()
        if isinstance(gpos,(tuple,list)):
            flip_time=win.flip()
            self.hub.sendMessageEvent("IMAGE_UPDATE %.3f %.3f"%(gpos[0],gpos[1]),sec_time=flip_time)
            reopened = True
        else:
            flip_time=win.flip()
            self.hub.sendMessageEvent("IMAGE_UPDATE [NO GAZE]",sec_time=flip_time)
2

I’ve never used it personally, but in principle I think you should be able to get pupil size by using tracker.getLastSample. Depending on your eye-tracker, that should (I think) give you a data structure that includes all of the info the eye-tracker is recording, including pupil size. I’d suggest starting by just calling getLastSample and seeing what it gives you, and going from there.

1 Like
3

Thanks for the tip! Yes it seems like tracker.getLastSample spits out a long list of eye data and cross refering to the hdf output, the 22nd item in .getLastSample is the pupil measure. However it seems like extracting this data is not as simple as the following codes I used:

eyelist =

for display in range(len(stimlist)):

for disptime in range(displaytime):
    
    reopened = False #track if eyes reopened
    
    # Get the latest gaze position in dispolay coord space..   
    gpos=tracker.getLastGazePosition()
    eyeinfo = tracker.getLastSample()
    print 'eyeinfo:', eyeinfo, eyeinfo[22]
    if eyeinfo[22] != 0:
        # If pupil size is bigger than 0, draw the 
        # background image and then the gaze_cursor.
        gaze_dot.setPos(gpos)
        gaze_dot.draw()
    else:
        blinked = 1
        print 'Blink occurred just before/at:', stimlist[counter]
        while reopened == False:
            gpos=tracker.getLastGazePosition()
            eyeinfo = tracker.getLastSample()
            if eyeinfo[22] != 0:
                flip_time=win.flip()
                self.hub.sendMessageEvent("IMAGE_UPDATE %.3f %.3f"%(gpos[0],gpos[1]),sec_time=flip_time)
                reopened = True
            else:
                flip_time=win.flip()
                self.hub.sendMessageEvent("IMAGE_UPDATE [NO GAZE]",sec_time=flip_time)

Executing print 'eyeinfo:', eyeinfo, eyeinfo[22] resulted in the following output/error:-
eyeinfo: 202 < type ‘exceptions.TypeError’ >
TypeError(““int’object has no attribute 'getitem””,)

Bear in mind that the 202 value is not a normal pupil measure value. It should generally be around 1900-2000~ when eyes are wide open, as is the case in the hdf output and eyeinfo output when recalled successfully. This happened right at the first frame of the experiment. Only when printing eyeinfo at the end of the trial/script was I able to see the list of eyeinfo items properly (hence able to cross compare with the data on the hdf file). It seems that eyeinfo doesn’t exactly generate values for all eye data when called for each time? Or is there something wrong with the above codes that is resulting in live data collection/extraction not done the right way?

4

Oh interesting. OK, so what’s happening is that at least some of the time, eyeinfo isn’t giving you a list at all, it’s giving you a single value, an int. The error is what happens when you try to look up an index of something that isn’t a list. For example, you’d get the same error if you did something like this:

x = 4
print x[1]

What print eyeinfo eyeinfo[22] SHOULD give you is first the whole list and THEN just the cell you’re looking for, so the 202 is just the raw value of eyeinfo. Why it’s the raw value of eyeinfo I have no idea, maybe that’s what happens when it can’t see the eye at all?

5

I just checked the iohub documentation and came up with two things:

  1. According to the iohub documentation (http://www.isolver-solutions.com/iohubdocs/iohub/api_and_manual/device_details/eyetracker.html), getLastSample will return an int (as it seems to be doing) if the method is not supported by the eye-tracker interface. That doesn’t entirely make sense because you said you can get eyeinfo when it is run at end of trial, so it’s not that it just doesn’t work with your eye-tracker, but I don’t know what would be different between end of trial and during the trial, except…

  2. I’m wondering if calling getLastGazePosition before getLastSample is clearing the sample buffer or making the eye-tracker panic because it’s being polled too quickly. getLastSample should give you those coordinates as well, among everything else, so you might try getting rid of those getLastGazePosition calls, setting gpos from the relevant cells of eyeinfo, and seeing if it starts behaving then.

Other than that, what eye-tracker are you using?

6

I’m currently using Eyelink 1000 Plus.

I managed to get to work again on it and fixed it by:-

  1. Removing gpos and only use eyeinfo as you suggested and

  2. Add an additional line of if isinstance(eyeinfo,list): above each of the if eyeinfo[22] != 0: and

  3. Drawing the gaze dot using gaze_dot.setPos([eyeinfo[12],eyeinfo[13]]) rather than gpos (12th and 13th item of eyeinfo is x and y gaze position respectively.)

My only concern here is that the eye tracker might not be consistently generating a list of eyeinfo values on every flip, thus the error that I encountered earlier? It’s only when I added step two did that resolve the issue.

7

Sounds like it’s working! You can check whether it’s getting data on every flip using that if statement. Whenever that returns false, add a line to your data file that’s all 0s or 9999s or something else obviously not real data. That will tell you how often it’s skipping. Alternately, if the eyeinfo array includes system time, you can see whether you’re getting data every 16.7ms or less often than that.

Also worth checking whether it returns an int when it’s not picking up the eyes at all, like if someone closes their eyes, which would be my guess.