Inconsistent format of output CSV files (delimiters)

Builder
1

I noticed that the format of the CSV outputs is not consistent across participants, for my experiment. I refer to the CSV file that gives trial by trial data, and that has ‘experiment’ in its name at the end.

One such file ends as follows (last few lines):

2,stimuli/04C1.wav,C1,2,1,807.0700684,807.0700684,0,816.3624878,816.3624878,0,807.0512695,807.0512695,0,,2,2,0,2.14199996,2.14199996,0,9.335264206,9.335264206,0,,816.3624878,816.3624878,0,,818.5146745,,67
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
extraInfo,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
session,1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
participant,2,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
date,2019_May_14_1543,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
expName,hSs_syntactic,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
psychopyVersion,3.1.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
frameRate,59.97727707,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

whereas another, for another run of the experiment, ends like this, with delimiters missing on the final rows:

2,stimuli/19C4.wav,C4,1,1.0,867.259765625,867.259765625,0.0,876.001708984375,876.001708984375,0.0,867.239990234375,867.239990234375,0.0,,2.0,2.0,0.0,0.7540000081062317,0.7540000081062317,0.0,8.769868850708008,8.769868850708008,0.0,,876.001708984375,876.001708984375,0.0,,876.7691273649689,,67.0

extraInfo
session,1
participant,03
date,2019_May_15_1145
expName,hSs
psychopyVersion,3.1.0
frameRate,59.92517496624865

In both cases, the experiment ran without problems to the end, and nothing was changed in the experiment (Builder) between the two runs.

This inconsistency leads to problem further downstream when I am processing these CVS files in Matlab. Any tips as to what might be causing this?

OS (e.g. Win10): Win10
PsychoPy version (e.g. 1.84.x): 3.1.0

2

Can anyone help?

3

Hi @tudor, can you share one of your csv files please?

4

Sure, please find both of the above-referenced files attached.02_1_hSs_2019_May_14_1543experiment.csv (13.9 KB) 03_1_hSs_2019_May_15_1145experiment.csv (20.0 KB)

5

Hi @tudor, these files open ok in my version of Excel. I wonder if your issue relates to the regional settings stored. For example, you may have semi-colon as your default delimiter. See the link below for changing settings [1]. Alternatively, I think you can covert text to columns by splitting the text in Excel [2].

[1] https://kb.itglue.com/hc/en-us/articles/115005306427-Changing-regional-setting-CSV-imports-
[2] https://www.extendoffice.com/documents/excel/1786-excel-split-text-by-space.html

1 Like
6

Thanks @dvbridges! The files open ok in Excel for me also; my problem was that their format is different towards the end of the file, as you can see with the delimiters etc. While this does not pose a problem when opening the file with Excel, it does when the files are processed with other analysis pipelines, which assume a consistent format across all input CSV files.

7

Oh I see. This should be related to what data is being saved, as defined in the Experiment Settings dialog in Builder. Below, I have “Save excel file” selected, which will give the format you have a above, and then “Save csv file (trial-by-trial)”, and this is saved without the extraInfo at the end of the sheet.

image

1 Like
8

THis makes sense, but what is surprising is that these settings were left (I think) as they were between the moments when the first and the second subject’s CSV files were produced. But maybe that’s not the case, as I see no other explanation why the change in format (presence/absence of delimiters) would otherwise occur.

Thanks for your help! :slight_smile: