Efficient image pre-loading

Coding
1

hello everyone,

i created an experiment but due to the large amount of images, the code crashes due to memory problems. Since it is an EEG experiment with fast presentation (one image per frame flip, or 0.177 ms), timing is very important and I need all images for a trial to be loaded before the start of a trial to avoid delays in the presentation. Total number of mages in the experiment is ~ 19200 (!) so I think it’s impossible to load them all before the experiment. I already tried to combine some images and save in pickle (in lists of numpy arrays) beforehand, and then in the main experiment loop to load images from the pkl files. However this does not speed up the process. Is there any way to make this possible without excessive loading time DURING the experiment? Thank you so much for any help

2

Have a look at my RSVP online demo and see it that helps.

3

Thanks for your quick reply! Unfortunately it doesn’t solve my problem, in the code you’re reading images directly in the main script, which is something I cannot do, due to the large amount (thousands) of images. Any further information is greatly appreciated

4

Presumably you don’t need them all at once. How many images are in a burst?

5

I don’t need them all at once indeed, but I need 3600 at once.

6

It depends how your code is set up now but if you are making a separate ImageStim for each of the 19,200 images, you could instead make only 3600 ImageStim, have a separate list of the file names, and then use the ImageStim.setImage() function to overwrite the image that was loaded before, or delete all of the ImageStim objects from the previous block to free up the memory before loading the ones for the next block. However, that will still leave you a loading pause in between blocks.

The only way to completely avoid a loading pause between blocks is to optimize the image file size as much as possible and/or get more RAM so you can load everything before the experiment. The memory errors are usually a pure hardware limitation, if you keep the individual images small and have a vast amount of of RAM (e.g., 32-64GB at least), in principle you could just load all of them before the experiment.

7

Hello and thank you Jonathan! It is a very good idea to overwrite the ImageStims, it works and is faster than each time creating a new one. In combination with a randomized selection of images varying across blocks, it still leaves a loading pause but now limited to ± 5 seconds between blocks.

I already optimized the size of the images to the max (±10kb) but indeed not enough RAM to load them all before.