Online experiment saved data was format misplacement

OS: mac OS Mojave 10.14.1 (18B75)
Psychopy version: Psychopy3.0.0b12

Hi everyone,

I tested one of my online experiment, and found the saved data was format misplacement(see the below picture). I guess this was caused by the text I want to present from the xlsx was forced to wrap. How to solve this problem? Is there any method not save this variable into the .csv file.

OK, I can see what is going wrong here. Please always provide a link to the study thatā€™s not working so we can investigate but, yes, the basic problem is that the data-file generator has not detected your new-line characters and identified them as being within the cell containing your text

Hi, @jon. Thank you for reply. Is there any way to fix this? For example, can I not store this variable in the generated .csv data. I can make these texts into pictures to present, but thatā€™s too much trouble. This is the git link. https://gitlab.pavlovia.org/yuanbo/moral_dilemmas_dm

I want to complete this kind of problems.

Iā€™m trying to make a very simple experiment where the participants have to read a text and then to answer to one question. I want to memorize the reading time.
https://gitlab.pavlovia.org/Lecture/textecoco

I have to say that in PythonPy, all works very fine! All the datas are perfectly saved!

The data saved in Pavlovia are in a wrong format. This is an example:

First when we have comma in the stimulus (in the text), it will make a column separation in the csv format file.

Question: is it possible not to save the stimulus in the csv format file? Or is there a way to specify another separator for the csv file.

Second: In the text to read, we have some new-line characters and this generate the same kind of problemsā€¦

Maybe the much simpler thing to do is to have an option which allow not to save the stimulus in the csv file?

Thanks for your help! The software is very very useful and in the next few months, I want to engage my students to use it!

Denis

HI @Fouc, thanks for the message. This is being fixed this week, so that the data outputs will match those you get from your local PsychoPy. That is, commas within variables will not split your data into several columns.

Hello,
Thank you in advance!
I have two more remarks:
First, will you fix the problem with the ā€œnew-line charactersā€ wich generate also new lines in the CSV file?
And another little problem: the name of the CSV file starts with the prefix PARTICIPANT and no more with the content of the field ā€œparticipantā€ in the dialog box wich start an experiment. I believe that was not the case a few week ago.
Again thank you!
Denis

Hi @dvbridges, I was just debugging a similar issue where I was trying to output the contents of an array to one cell in the recorded data spreadsheet. An array will break a single cell up into many columns, mangling the output. I worked around this issue by recording the array as a string joined by underscores (using the join() method) but is there a plan to implement a change to data recording such that an array can be recorded to a single cell without this workaround?

PsychoPy should ideally adhere to the csv standard and enclose all cell contents with double quotes, so that they can contain delimiters and end of line characters without disturbing things. Iā€™m not sure if that is currently happening, but could be specified in the pandas export functions I think we use?

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_csv.html

So there was a problem with how new lines were interpreted when the conditions file was imported, where the newline characters were escaped. I have submitted a pull request to fix the local Python version, and we can talk to Ilixa about what they can do with PsychoJS

@Fouc, I just tested this, and PARTICIPANT was only used in the filename if there was no entry in the participant field. When I give the participant field a value, the value was used in the filename. Is that the case with you?

No. I have always PARTICIPANT even if I put an entry in the field.

Sorry, it was my fault for the PARTICIPANT in the name of the csv fileā€¦
In let expInfo = {ā€˜participantā€™: ā€˜ā€™, ā€˜sessionā€™: ā€˜001ā€™};, I wrote
let expInfo = {ā€˜Participantā€™: ā€˜ā€™, ā€˜sessionā€™: ā€˜001ā€™};
with a lowercase ā€œpā€, it works perfectly wellā€¦

But not for the precedent problemā€¦

Thanks @Fouc, yes I have fixed the problem in Python re: newline chars obeyed when importing text from conditions file, and looking to get the fix done for PsychoJS libs.

Hello,
Sorry, but there is something that do not unterstand very well. Because of ny english, probably.
The problem was not with the input file, but with the output file, in a CSV format. Ant it still thereā€¦
But I wish you a very good year and I suspect it will be fructuous for all our experiments.
Again thankā€™s
Denis

Hi,
I want to say to all the developers that all the problems that I had encountered are fixed! Thanks to all!
I needed to start a new experiment with the new version and to synchronize it on Pavlovia.
No more problem with the save of the CSV file!
Encore merciā€¦

Because I use TextStim object, I encountered some options that are not generated by PsychoPy, per ex ā€œalignHoriz : ā€˜leftā€™ā€, which is very useful for the presentation of texts.
I donā€™t know where to put these kinds of observationsā€¦
Thanks again!
Denis

Hi, this is an old post i know, but this is the problem i am having with my pavlovia CSV output data, it is all very misaligned and hard to analyse when using the pavlovia output but all ok when using the builderā€¦was the PsychoJS ever updated, as at present the output is not matching the builder structure. I am using the most up to date builder version.

2 Likes

Weā€™ve got a similar problem. Funnily enough, the output is perfectly fine when downloaded immediately after piloting (when being asked whether to store/open it after exiting the piloting session), but when downloaded via the ā€œDownload resultsā€ button, the text presented within the experiment is printed in the output, sometimes muuuultiple times (in one case the same text is in the output file 210 times - it is not presented within the experiment 210 times)

1 Like