Could someone give me a little more information about how the moviestim class chooses a particular frame on a call to draw?
I’m doing some eyetracking with psychopy and pygaze. For data visualization I wanted to be able to present a marker for the eye position at each frame. However this requires me to map each measure from the eyetracker to the frame it was sampled during.
I noticed that using the moviestim class the number of calls to draw is not the same as the number of frames in the video, and in fact isn’t even consistent from run to run. Looking at the source the code says the current frame which the moviestim draws is “automagically determined”. I was hoping someone might be able to elaborate on what that means?
I think it is based on elapsed time and the frame rate of the movie. eg your PsychoPy code might be drawing stimuli to a display running at 60 Hz but the movie file is recorded at 30 Hz. That means that each frame of the movie would be drawn twice.
“Automagically” means that it isn’t PsychoPy itself that is handling the selection of video frames: that is handled in the background by whatever movie backend is being used. There won’t always be the same integer number of times that each frame is displayed: e.g. imagine common scenarios like a video at 29.97 Hz being played on a fast LCD monitor running at 144 Hz.
Your visualisation playback might be happening on a different monitor with a different refresh rate. Maybe it’s best to just think of events happening at a point in time during the movie rather than on a specific frame, as there will potentially be a bit of slop of plus or minus 1 video frame at any point, depending on the ratio of video frame rate to display refresh rate.
Thank you Michael this is helpful!
We do have a time stamp for each eye sample, so if there is a way to do an accurate reconstruction with that data that’s an option. But I think, if possible, the frame/sample mapping is better. When we make the visualization, we add a point indicating the eye location to the individual frames of the video. If we don’t know what frame was at what time during the actual data collection, then I’m not sure how the time information helps us? Unless there is a way to reconstruct which frame each sample was taken given it’s time stamp, the frame rate of the movie, and the display rate?
If we know the frame to sample mapping then it is simple.
Is there a way to report the current frame at each step in the movie?
I don’t know that we get access to a frame number per se, but you can actually record the actual frame being displayed at any point, using the getMovieFrame function:
https://www.psychopy.org/api/visual/window.html
but that might be overkill for you. Realistically, you’ll be doing the same thing at the analysis stage as the movie playing code does. eg an event happens at 10 s into a 30 s movie that is encoded at 25 Hz but played on a 60 Hz monitor. Select the frame that corresponds to which one would be displayed at that time. At worst, you would be out by one from what was actually shown.
eg in this case, there isn’t an integer ratio between the frequencies, so some movie frames are shown for 2 screen refreshes and some for three, to maintain a mean duration of 60/25 = 2.4 screen refreshes per movie frame. The movie back end takes care of that frame selection and we don’t know how deterministic it is (eg the precise frame shown at a given time could even vary from run to run).
Then a natural question arises as to whether you are “just polishing the cannonball” by trying to record the exact frame presented at an exact time. If your analysis is fixation based, then the duration of that spans multiple movie frames anyway. If it is saccade based, then the sampling rate of the eye tracker is crucial. If is only 60 Hz, then it’s resilution is worse than the monitor itself, as it takes several samples to determine the onset and offset times.