URL of the experiment: Pavlovia
Description of the problem: We have been collecting data for over a year. I downloaded the most recent results yesterday (results are saved to the database and then downloaded after a certain number of participants) and found that my CSV file has changed format. This is what my results csv file used to look like (consistent across many downloads):
This is what the new spreadsheet looks like (I redownloaded the results to see if it was an isolated error and it isn’t):
It looks like the ISI column has been expanded into individual cells (using the comma delimiter) and shifted over all the other columns. I verified that my code had not been changed, so it appears to be potentially related to an update on Pavlovia’s end?
Does anyone know how this could have happened or how I can return my csv file to what it was previously? Has anyone else experienced this? Or does anyone have suggestions on how to clean my datasheet to remove these cells?
Thanks in advance!
Dear Jen,
I am not sure what exact changes to pavlovia that were done that might have done this. I know one thing you could do is open up a blank excel file. Then click on Data-> From Text/CSV
Import one of your incorrectly formatted files and try messing around with the different functions. It may have been excel incorrectly reading the csv file as one file origin code compared to your original ones, you could also setup a custom delimiter option to mimic the original data output. If neither of those options work (and it wasn’t pavlovia’s fault) then the only options that may be left are making a python script or cleaning data by hand
.
Issac
Hi Issac,
Thanks for your response. I tried messing around with inputting the data and wasn’t able to find a complete solution, but something that will help if I have to manually clean the data. So, thank you for your suggestion!
I’ve continued to look through my data and found that another list of response timings that is in the same format (all values are enclosed within square brackets, which are inside double quotation marks) didn’t get expanded.
I also compared the incorrectly formatted files to the original format files using the TextEdit app and found that there were no apparent differences between the two files other than the order of the columns.
This made me wonder if the list was being expanded due to the ISI column being the first column. So, I edited the incorrectly formatted csv through the TextEdit app, created a new column header and added a comma before the first row’s list. Then tried to import this and found that it just moved the start of the list over to the second column, but it was still expanded.
I appreciate your help on this 
Hello
This looks like an Excel import problem. Check the files with a simple editor and analyse the data with a proper statistik software. R could handle this without problems.
Best wishes Jens
Hi Jens,
I agree that it is partially an Excel import problem. It was also causing problems for my python code that was importing the data (I just used Excel to view the data after my code didn’t run). I appreciate your suggestion to use R, it was able to import the data correctly.
I was able to get the data to a usable state; my remaining question is why, all of a sudden, the order of my columns changed when they were consistently downloaded in a certain order for over a year. I last downloaded the data in September with no issues.
Cheers!
1 Like
