Hi, first poster here. I would like to report some issues with movie playback using moviestim3/moviepy. More specifically:
- The first frame (containing movie.draw() and win.flip()) takes very long to display, possibly messing up timing of video presentation.
- After seek(), the movie does not start from the new position, instead, a few frames from before the seek are displayed, after which the video jumps to the new position.
- Also after seek(), the video is played at a higher frame rate, as if to “catch up” with some kind of mistaken timer.
My video was playing choppy sometimes, playing too fast other times, so I did some testing in which I measured frame draw and total cycle timings.
Disclaimer: I’m fairly new to psychopy and python itself, so my methods, thinking, conclusions may be inherently mistaken. I’m happy to learn new things so please be patient and explain.
My setup: I’m using psychopy 3.2.4 library in python (not the standalone); moviepy 1.0.1, pygame 1.9.6 on win10 x64, GTX 1080Ti, i7-8700K.
I used the following code for the tests:
import psychopy.core, psychopy.visual, psychopy.event, psychopy.monitors
import time
import numpy as np
import matplotlib.pyplot as plt
# setting up monitor, opening window
monitor = psychopy.monitors.Monitor('LG32UD99W', width=70, distance=40)
monitor.setSizePix([3840, 2160])
win = psychopy.visual.Window(size=[2560, 1440], monitor=monitor, units="deg", screen=0, color=-1., colorSpace='rgb', fullscr=False, allowGUI=False)
# 2 movies: 720p or 1440p, both have no audio track
movie = psychopy.visual.MovieStim3(win, '720p.mkv', size=(1280, 720), noAudio=True)
# movie = psychopy.visual.MovieStim3(win, '1440p.mkv', size=(2560, 1440), noAudio=True)
# The 3rd test included this seek command:
# movie.seek(150)
movie.play()
# loop 300 win.flip() cycles, measure movie.draw() time and cycle time
n = 300
frametimes = np.zeros((2, n), dtype=float)
for i in range(n):
a = time.perf_counter_ns()
movie.draw()
b = time.perf_counter_ns()
win.flip()
c = time.perf_counter_ns()
frametimes[:, i] = (b - a, c - a)
# cleanup
movie.stop()
win.close()
# display frame time plot
ft_ms = frametimes / 1e06
fig = plt.figure(figsize=[14, 4.8])
plt.suptitle('720p')
plt.plot(ft_ms[0, :], label='draw')
plt.plot(ft_ms[1, :], label='flip')
plt.xlabel('iteration #')
plt.ylabel('time (ms)')
plt.legend()
plt.show()
Testing results (only 3 shown here, did many more with similar results):
- 1280x720, 25 fps movie
Most of this image is as it should be: cycle total times (labelled as flip) correspond with the monitor’s refresh rate (1/60); draw times zig-zag because the movie frame rate is 25 so in about half the cycles the draw() function reuses the cached last frame (moviepy keeps track of the elapsed time and only provides new frames when the time is right); decoding new frames takes some time, but when the last cached frame is used, the draw delay is very low. However, the first frame takes very long (red circle). It is not always the first frame, though; in some tests it comes a few cycles later. Why does it happen? Is there some kind of initialisation going on? Is my testing method faulty? I haven’t digged very deep into the code but I’ll try to find out. I’m also not 100% sure that this affects actual display timing, but it could.
-
2560x1440p, 25fps movie
<< I had an image here too but new users are limited to 2 / post so this one had to go. >>
Same test with a larger-resolution movie. Basically the same result, with the first frame delaying long, the rest as expected. Frame decoding times take up nearly all the available time in a 1/60 s cycle, though. The total decoding time is not prohibitive for this resolution at this refresh rate, so it would be nice if there was a way to spread out the decoding over frames - something like a frame-server that takes care of pre-loading and decoding images, and serving them as needed. That would also allow even faster refresh times at this resolution (a topic brought up in other threads in this forum). -
Same 1440p movie, but adding a seek(150) before play()
The large initial frame delay is there, but the main point of this figure is the part after the initial peak (red ellipsis). There is no zig-zagging during the first ~15 cycles. This, I believe, is some kind of timing problem, perhaps the movie thinks it’s lagging behind so the frames are served at every cycle to catch up? A brief look into the code tells me that both psychopy and moviepy have their own way of calculating frame timings, but I haven’t spent time on understanding why this happens. Why is this a problem? Because if I ever do a seek() in my experiments (for example to randomise which part of a longer video would play during a trial), the video would play very fast initially, which is unacceptable.
My conclusions, please let me know if you agree:
-
Does the initial long frame delay affect trial display times in a bad way? The duration is not constant, so if actual display times vary like this, that’s not very nice. I know that for best timings one should use a photodiode to check actual display times, which I will do, but if this delay is there, it should be eliminated nevertheless.
-
As it is now, I think it’s better to avoid any use of seek(). It was not part of my test but I mentioned before that frames from before the seek leak into the video, and then the video image jumps to the new position. Plus, the video can play at a faster rate to “catch up”. In a streaming application this faster playback might be desirable, but in an experiment it is not acceptable. Delays can be measured with a photodiode, but the video rate should be reliable. Obviously, a 30 fps video would be better for a 60 Hz display, but the small rate difference is tolerable.
