Hello,
We run our programs 24/7 so the animals have regular access to them. Occasionally the programs will crash due to power issues in the building, etc. When this happens, there is no data file which is a BIG problem, as thousands of trials are just lost. Is there any way to change the settings or add code so that the data file is written and updated after every trial instead of when the program ends or is closed out of? I am really desperate for help on this and have posted this question a few times now. ANY advice would be welcome!
Thanks so much,
Brooke
Hello
do you run the experiment offline or online? Offline, PsychoPy saves the data trial-by-trial when the option "Svae csv file (trial-by-trial) is checked. This the default.
Since this is the default, I have no idea why it fails in your case when there is a power shortage. The only thing I can imagine is that file writing and low power might interact in a way that “empties” the file that has been written so far.
If you are running the experiment online, check the option to save incomplete data sets.
Did you try to “simulate” a power shortage and see what happens?
What about buying an uninterruptible power supply? Uninterruptible power supply - Wikipedia
Best wishes Jens
Hello,
The programs are not being run through pavlovia, just through the .py shortcut on the laptops. The power issue was just one example. We have a lot of storms in the area that cause whole building power outages so there’s not much we can do about that. But anytime the program crashes there is no data file saved. After thousands of trials, occasionally the program will crash (memory usage issues, laptops auto-update, etc). Others in the lab use python to create programs and they will also have the occasional crash but the data file is still created.
I notice that if I start the programs, do a few trials, minimize the program, and check the data folder in the experiment folder, there isn’t a data file created. The data file is only created if the experiment ends, or esc is used to close the program. So if the program is forced to close, those criterion are never met to create the data file. I am hoping there is some way to have the data file write trials after each trial occurs, instead of waiting until the program ends or is closed out.
Thank you so much for your help.
You can write the data file in a code component by calling:
thisExp.saveAsWideText()
With your desired filename. You could do something like if (trials.thisN % x) == 0: to save every X trials.
1 Like
Thank you so much for your response! That seems to save every trial as an individual csv file, which would be a big issue considering we often get 10,000+ trials a day. Is it possible to have it just append the data to the existing file on every trial instead of creating an entirely new file?
If you supply a file name when calling saveWideText, then it’ll save to that file - so you could provide the same file name each time to continuously overwrite it (meaning just the one CSV file)
1 Like
I provided a filename and it just created filename.csv, filename.csv (1), filename.csv (2), etc.
I ended up putting this code in the begin experiment tab of my first routine:
import os
import datetime
current_date = datetime.datetime.now().strftime("%d-%m-%Y_%H-%M-%S")
folder_name = "backup"
datafile = os.path.join(folder_name, f"{expName}_{expInfo['monkey']}_{current_date}.csv")
and then this code in the end routine tab of the last routine:
if (trials.thisN % 1) == 0:
thisExp.saveAsWideText(datafile, fileCollisionMethod='overwrite', delim='auto')
And this seems to be working. Thank you so much for your help! I really appreciate it! I really like using psychopy since I also occasionally put experiments online for human testing, and because I am not an expert programmer by any means. I was getting worried I was going to have to switch to using python like my colleagues. But this should fix the last problem I was having!
Ah, I forgot that the default for fileCollisionMethod was ‘rename’, my mistake! Yep, setting it to ‘overwrite’ is exactly the right solution. Glad this is working for you now!
1 Like
