MemoryError / Too Many Files Open

Coding
1

Hello!

I am trying to present a series of 150 images and videos while collecting a participant response (150 trials, each with an image, video and response).

The problem is, that on trial 62 everything crashes with a “MemoryError” or “too many files open”

I think the problem is with MovieStim3?

mov = visual.MovieStim3(win, t.videoFileName, pos=((layout[‘spacing’]+layout[‘video_size’][0])/2,0), size=layout[‘video_size’],loop=True)

I am changing the name of the movie to be loaded on each trial.

Every new movie, though, PsychoPy creates a new process and the memory load increases on the computer (as assessed by the task manager). Until death…

I am thinking that psychopy is not clearing the video from memory after the video is done running.

After the video is over, the memory load does not go down and the process does not end. So the number of running processes and the memory load continue to build over the course of the task.

We are trying to run about 150 trials (using 150 videos) and the program crashes from a memory load error or a ‘too many files open’ error on trial 62

If the error is happening on trial 62, it shouldn’t be a preloading problem. My guess is that it loads each video as it is played but doesn’t release the memory afterwards.

How can I get PsychoPy to clear the memory of the video after it runs?! I have tried things like Del and Close but they don’t seem to be working.

Thank you so much for reading and if anyone can help it would be greatly appreciated!

I am using

PsychoPy2 Coder (IDE) (v1.90.2)

64-bit Operating System, x64-based processor

Traceback (most recent call last):

  File "C:\Users\lab-user\Documents\GitHub\CERES\CERES_EXP_PJR.py", line 431, in <module>

    sequential_trial(i)

  File "C:\Users\lab-user\Documents\GitHub\CERES\CERES_EXP_PJR.py", line 241, in sequential_trial

    t.showVideo()

  File "C:\Users\lab-user\Documents\GitHub\CERES\CERES_EXP_PJR.py", line 86, in showVideo

    mov = visual.MovieStim3(win, self.videoFileName, pos=movie_loc, size=movie_size)

  File "C:\Users\lab-user\Desktop\PsychoPy2\lib\site-packages\psychopy\visual\movie3.py", line 134, in __init__

    self.loadMovie(self.filename)

  File "C:\Users\lab-user\Desktop\PsychoPy2\lib\site-packages\psychopy\visual\movie3.py", line 180, in loadMovie

    self._mov = VideoFileClip(filename, audio=(1 - self.noAudio))

  File "C:\Users\lab-user\Desktop\PsychoPy2\lib\site-packages\moviepy\video\io\VideoFileClip.py", line 81, in __init__

    fps_source=fps_source)

  File "C:\Users\lab-user\Desktop\PsychoPy2\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 72, in __init__

4.3545     WARNING     Monitor specification not found. Creating a temporary one...

4.3563     WARNING     User requested fullscreen with size [800 600], but screen is actually [1920, 1080]. Using actual size

    self.lastread = self.read_frame()

  File "C:\Users\lab-user\Desktop\PsychoPy2\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 147, in read_frame

    result = np.fromstring(s, dtype='uint8')

MemoryError
2

Hi,
Just to upvote this issue, I’m dealing with the same problem - 120 video files requiring a judgement and the experiments stops after about 30 videos and says “too many files open” (on both a windows 10 running 1.90.3 and Mac 10.13.4 running 1.90.3).
Thanks,
Mark

3

It is what it says on the box. MovieStim3 objects are gigantic RAM hogs, especially if they have audio. If you’re loading one per trial, my recommendation would be to make sure to delete each one at the end of the trial and try to free up the memory it was using. Other than that, all you can really do is get more RAM or shrink the movie files by reducing their resolution or framerate.

See also: MemoryError or Too many open files with MovieStim3

4

Thanks - how does one delete a file after using it? “Garbage collection” is automatic in python isn’t it? Is there a way to force it to flush the allocated memory for a video that isn’t needed anymore?
Thanks,
Mark

5

Have you tried a newer version of PsychoPy?
Some MovieStim3 bugs were fixed last summer.

Try PsychoPy3.

6

Hi,
Thanks, yes, I’ve tried the latest version (it didn’t solve the problem), but it also caused problems with audio that were solved by going back to 1.9.

7

Shot in the dark here, but maybe the moviepy module needs updating?

8

Hi,
Possibly, but I think jonathan.kominsky has correctly identified the issue - that the RAM is being used up. The link he included showed a way (I believe) to unload the videos, but it seems to be a Coder specific solution, not something that can be implemented in Builder (that I can see).
Thanks
Mark

9

I imagine that you could just put those del statements in a Builder code component in the “End routine” tab, referring to the name of your movie stimulus.

If you try that and it works, please let us know.

10

Hi,
Thanks for your help. I tried adding the specified lines in a code element and got an error message about the “del” lines. So I tried just removing those lines (which I didn’t understand anyway) and simply use the “unload()” line. It seems to be working like a charm - no errors and no running out of RAM.
Thanks,
Mark