My experiment crashes when it is run for a long-ish time (c.25-30 mins). In a previous post (Crashes after 30 mins/ wavs have clicks/ can’t use MIDI files ) I considered the possibility that it was because of the amount of data being saved. However I’ve now got it printing the data to an excel file after every 5 blocks, and resetting the experiment handler after it’s done this. I’ve noticed that if I complete the experiment quickly, I can get up to 10 blocks before crashing (with 2 excel files thus saved), but that if I complete it slowly it won’t even reach 5 blocks. As such, I don’t think it’s an issue with the amount of data being saved…
Trouble shooting, what other cause might there be for this? Perhaps something to do with clocks? Has anyone else had the experience of their experiment crashing after a certain period of time?
It’s basically an experiment that plays participants a series of sounds and asks them to recognise them.
Have you checked whether there is a memory leak? Open the task manager and check whether the amount of RAM used by your experiment increases over time. Once the RAM is filled up, your experiment will crash and the the RAM will be released (if your issue is caused by a memory leak).
Thanks for that. Yes the memory usage does seem to be continually ramping up as I use it. When it eventually crashes it crashes the whole computer, so it’s relatively severe!
Any idea as to what might be causing a memory leak? I tried writing in code that saves the data from the experiment handler, then resets the experiment handler, every 5 sets of trials - doesn’t appear to have fixed anything…
It seems very unlikely that it is the accumulation of data that is causing the problem. For most people, the experiment handler is just storing a relatively small amount of text/numeric data per trial, which isn’t going to tax your computer’s memory.
But somehow you seem to have isolated the memory memory leak to the ExperimentHandler? If that is the case, I wonder if you are inadvertently storing some of your actual sound stimuli within the ExperimentHandler, rather than just the names of their associated files? I tried to look at the code for the minimal reproducible example you posted here:
I just ran if for 20 minutes before pressing escape to exit, hoping to collect some error messages, but didn’t get anything?
Weirdly it seems like I got some from a previous attempt though:
########## Running: /Users/sam/Downloads/AP_experiment_1/AP_exp_1.py ###########
pyo version 0.8.6 (uses single precision)
2018-06-09 17:58:58.264 python[3307:160605] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/k8/grc8crt12fg_6cq4rwvvcl7r0000gn/T/org.psychopy.PsychoPy2.savedState
14.4518 WARNING t of last frame was 20.02ms (=1/49)
15.8141 WARNING t of last frame was 34.33ms (=1/29)
15.9307 WARNING Couldn’t measure a consistent frame rate.
Is your graphics card set to sync to vertical blank?
Are you running other processes on your computer?
18.8743 WARNING t of last frame was 2943.81ms (=1/0)
19.3572 WARNING t of last frame was 19.91ms (=1/50)
20.6185 WARNING Multiple dropped frames have occurred - I’ll stop bothering you about them!
Any thoughts?
This is the actual code, in case that’s of any help (it’s prob too much info to be useful…) - AP_exp_1_simplified.py (48.5 KB)
Those are warnings about graphics underperformance. We really need to see actual error messages.
Not too much point looking at the code until then, as these error messages might help zoom in on the source of the problem. But on an initial pass, this looks like a Builder-generated script. Are you just adding a code component to a Builder experiment, or have you done more major modifications to the script?
I think the reason I don’t get error messages is that when it crashes it doesn’t just crash the experiment but actually crashes the whole computer losing everything… any thoughts?
I used the builder code as a starting place, but that’s about it - it’s mostly new code…
Anyone else will struggle with this too: given the size of the code and the lack of any error messages, it is basically impossible for anyone to suggest much when working remotely.
The usual suggestion here is to get things down to the absolute minimum in which you can reliably produce the error. I’d suggest working with a standard Builder file, with code added via a code component, rather than directly modifying the script. There’s just too much that can go wrong with manual edits like that, making things impossible to narrow down.
Sure thing - thanks for your message. I wasn’t expecting you to get a lot from the code itself seeing as there is a lot of it, I only really posted that here cause you asked.
The issue is that you’d like to see the error messages, but I can’t get error messages cause it shuts down the whole computer when it crashes. So question 1 is: is there a foreseeable way in which I can get code that crashes the whole computer to generate error messages, before it does so, that you’d find helpful?
Question 2 is, presuming I can’t generate these error messages for you, if I’m going to troubleshoot this what are the standard kind of things that might cause a memory leak like this? (I take your point about going through it bit-by-bit etc, but it would be nice to have some inkling of a clue for what sort of things I might be looking for…)
To be honest, it’s highly unusual to get a crash that brings down the whole computer. Modern operating systems are designed to constrain issues like this so that only the affected process gets terminated. So we’re flying a little blind here.
Similarly, on your second issue, memory leaks are also pretty rare in Python, as memory allocation is something that gets abstracted away from the programmer. It is possible to build long lists of large objects, though, and not get rid of the references to them after they are used. I seem to recall that you had a very long list of sound files? It would be worth checking that you aren’t creating a new sound stimulus for each, but instead just create a single stimulus, and point it to a new sound file each time as required. That ensures that you aren’t ever maintaining a reference to more than one sound object.
condition_selector is a function that I’ve written that takes a dictionary containing all of the possible conditions, and then picks a few of them for me. This is ignorable I think, but for interest’s sake here is the function:
def condition_selector(a_conditions_file, octaves, keys):
new_conditions = []
audioStimuli = []
shuffle(a_conditions_file)
last_corrAns = 'blank'
for hh in range(0, octaves):
shuffle(keys)
while keys[0] == last_corrAns:
shuffle(keys)
for nn in keys:
for jj in a_conditions_file:
if jj['corrAns'] == nn and jj['corrAns'] != last_corrAns and jj['audioStim'] not in audioStimuli:
new_conditions.append(jj)
audioStimuli.append( jj['audioStim' ] )
last_corrAns = jj['corrAns']
return new_conditions
In terms of actually summoning each audio file I just use the following within the loop for each individual trial:
audio_stimulus.setSound(audioStim, secs=-1)
I think that resets the sound for audio_stimulus, thus clearing out the previous sound from it. As such I don’t think there’s anywhere where the sound files could be cumulating, causing a memory leak? What do you think? It’s a logical enough idea for explaining the memory leak, as I am indeed handling a lot of files, but I think each of them replaces the last on each iteration of the loop.
That sounds like you are doing the right thing then.
I think from here you need to look at using a memory profiler to show you the memory usage for various objects in your code. I’m not too familiar with them but there are a number of alternatives to use. If you are running this code from with the standalone PsychoPy, you will probably want to choose one that you call from within your code, rather than use it to run your code.
I had this issue, and it turns out that the experiment was logging too much information about the data itself. If you go to “experiment settings”>“data” and then uncheck “Save log file” this should help keep the experiment from crashing.
