psychopy.org | Reference | Downloads | Github

Pavlovia credits reserved, no data

Hi,
I did a second trial run with 3 friends that assured me that the waited until the test told them that it was safe to close the window, but i only got reserved credits and cannot access their data,

What have I done wrong?

1 Like

@cami94 most people’s experiments are certainly running fine to the end and saving data, so if yours isn’t we would need to work out what’s different. Is it a long study? Is there anything unusual about the data being stored, or the type of study? Are you redirecting subjects somewhere at the end? Basically, try to tell us anything that you can think of that distinguishes your study from basic examples that are working.

@jon it consists of many small parts, i think it’s about 13 routines, but all part’s is quite simple,
is there anything i can send you? unfortunately I have very little experience within this field… yesterday i had 154 reserved credits and today 22, and I know that a lot of people has done the test today, so I suspect that a lot of data has been lost…

Please let me know more in detail what I can send you so that you can help us!

no redirection, basic IAT-test

Hello @cami94,

I have just run your experiment. The log files were correctly saved in the data directory. As a matter of fact, I was your third participant today, and you had 6 yesterday.
The experiment results are saved in the database, since your experiment is configured for database saving rather than .csv. That is perhaps the cause of your concerns.

For instance, this is the last entry from my run:
{ "_id" : ObjectId("5ea18f10130a1fc76c1f4646"), "__projectId" : 20044, "__experimentName" : "openIAT", "__participant" : "ANONYMOUS", "__session" : "001", "__datetime" : "2020-04-23_14h46.53.043", "key_resp_4~keys" : "space", "key_resp_4~rt" : 3.6113399998284876, "participant" : "ANONYMOUS", "session" : "001", "date" : "2020-04-23_14h46.53.043", "expName" : "openIAT", "psychopyVersion" : "2020.1.2", "OS" : "Linux x86_64", "frameRate" : 60 }

Could I ask you to go to your experiment page from your dashboard and click on the Download Results button? You should be able to access all of your results in that way.

To change the way the results are saved, go to your experiment page and under Saving Format (on the right hand side), click on CSV. Your results will then be saved in .csv files.

Cheers,

Alain

Hi Alain,
thank you for checking on this, I have however 5 new reserved credits most of which was from today, and I I have more than 10 people that wrote to me saying they took the test today. All of whom I asked if they had waited until the screen said “thank you for your patience”. then gone blank.

I currently only have 36 replies accourding to my dashboard, out of which i am positive that at least 14 is my own and from our send test trial as I suspected that the answers weren’t saved correctly…

And it still doesn’t explain the more than 150 reserverd credits from the day befor yesterday, is it really no way that you have a backup of those that we can double check? that represent almost half of the replies we hope to receive…

when i download I only get from the 34 .csv objects with data, out of which 1 is “all data”

PLEASE let me know if there is anything else I can do or provide you with to take a look at this,
I am really afraid that we are loosing data that we will not be able to collect again

Hello @cami94,

I am very surprised by the disconnect between what your participants are reporting to you and what the pavlovia logs are showing.
For instance, on the 23rd of April, only 9 sessions for your experiment were opened. 3 were completed, 1 was aborted (probably pressing ESC), and 5 hung (probably just closing the browser tab). That is somewhat different from what was reported to you.
From yesterday’s logs, 6 people started your experiments, two completed it, one aborted it, and the other 3 just gave up and closed the tab. At least as far as I can tell.
The 150 reserved credits are from 50 people starting your experiment but not finishing it on the 20th and 109 starting but not finishing on the 21st. Which participant service are you using? Sona perhaps?

Your experiment is long and repetitive: in my view, it is entirely possible that people get bored half way through and give up.

Do you have a way to ask someone to take the experiment for you, and tell me what happens then? There is not much I can do on my end since, as far as I can tell, all is working well. And, as you mentioned, you can run the experiment without issue and so can I.

Alain

Hi Alain,
As you mentioned this does not not add up, for example i know that at least 4 was completed yesterday as i was sitting beside or were on the phone with the participants, but i could only access the two completed. The test ran as usual, no glitch and all of the participant got to the point “thank you for your patience”

I have not used any participant services, i have shared the link on social media in groups that represent our target group.

is there anyway to access the data that was started but not finished? aborted test data is stored localy when doing it in builder, so it should store that data on the server as well? if we could access that we could identify where the error is - my suspicion is that it might be related to the ok button on the “thank you for your patience” window, making the test valid and complete but just not stored.

I have a hard time believing that between the 159 people did not complete the study between the 20th and 21st as they before the test had been informed of the design of the test and the time it will take

Please let me know if there is ANYWAY to access this data

I can see that participants started your experiments at those times:

2020-04-27 10:10:28 +0000
2020-04-27 10:18:29 +0000
2020-04-27 15:39:33 +0000
2020-04-27 16:15:45 +0000
2020-04-27 19:14:43 +0000
2020-04-27 19:34:19 +0000

As far as I can tell, only the last two participants completed it. At one time were you sat with your two participants?

Incidentally, when the first session started, the experiment’s saving format was DATABASE. I can also see that a request to change the experiment saving format from DATABASE to CSV was made later on at 2020-04-27 18:22:42 +0000. Did you do that?
Did you notice that you have one participant result from the DATABASE for 2020-04-26, and several from 2020-04-23, among other dates?

There is no way to access the data of sessions started but not finished at the moment, unfortunately.

Looking at the code of your experiment, it appears that you are checking whether the ESC key is pressed even in the thank you routine. If the participant presses ESC at that point, it will abort the session and no result will be saved. That is however not the issue here since we do see the aborted sessions in the logs and you did not have any yesterday.

I have no difficulty believing that over a hundred participants (students, I imagine) decided to give your experiment a go but were not motivated enough to finish it. It is already tricky enough to get participant to finish when there is a reward, but in the absence of a reward, and running your experiment completely anonymously over the internet, it is very easy for them to just go away as soon as they are bored.

Alain

yes it was I who changed this, as per your previous comment that it might be the cause, i thought i would try to change it to .csv and see if it made any difference - but it didn’t.
Which do you recommend that I use?

I was present/ on the phone with the 4 latest.

I would say that maybe 75-100 persons are students, the rest is actually business consultants all over the world from a big company, so I do still have a hard time believing that that many just decided to shut down the test…

FYI, we think that we have identified the problem, but we still have no solution.

we have noticed that it is most likely the choice of browser, although the test works in all browsers, when we have asked people to printscreen their last step some of them did not get the last dialog box that says “closing the session please wait a few moments”, but only a white screen.

Thus they close the window too early and their participation is not saved as a valid result and only reserves a credit.

Is there any chance to fix this?!

Hi, I think I have a similar issue. So far, I appear to have 51 individuals who started the experiment. Of these, I only have 7 data stored, 5 aborted, but 41 are on reserved credits…isn’t it a very high proportion?
Thanks!

Hi, Elena

I am finding the same problem right now. Did you find a solution?

There is now an option to save incomplete results (at a cost of the credits).

What is the participant expected to do in the final trial?

Hi,
I have it set to save any aborted sessions, however, when I download the results, the excel file appears blank in all aborted sessions.
The participant is indicated to press “space” to exit the experiment and wait for the dialog box with the confirmation of the data being uploaded correctly.

Hi Sarah,
my solution is that I only keep full data…
Reserved credits get re-set after 24 hours, and those are likely the ones that stopped the experiment before finishing it (which are A LOT, but may depend upon my experiment, which is a bit long).

Hi, Elena. I can understand for an experiment to have participants that give up, however at the beginning of the data collection I asked friends and peers to fill, so I find it hard to believe that out of 20 people, 12 got bored, even when they knew how long the experiment would take.

Dear Sarah,
my experience is that data do not get stored for a series of reasons…Some of my participants, for example, actually reached the end of the experiment but failed to attend the latest writing “data saved” and press on ok. Others were given an error message saying that data couldn’t be stored for reasons I still do not fully grasp (but I think if participants re-start the experiment on the same computer after they quit, this causes troubles).
What was your participant’s experience? Their feedback could be quite important to understand if they really made it through or actually quit or had any other problem.
I had over 300 participants starting the exoeriment, but only 80 complete data, so the ratio is very low, and most of them actually quit.

I actually have a similar problem on my experiment. I tried running it as “running” and when I tried to download results, it gave me the files in the “data” folder from my GitLab and not the actual data from the “running” session.

I have since deleted the folder, but nothing yet has shown up.

URL of my Experiment if you are interested: https://pavlovia.org/mtruong/test5

I have a similar problem here…I noticed that some people’s data was not saved properly (some were saved but then after I open it it’s empty), even though those are the people I have full trust that finished the study, but maybe did not quit properly or something to cause the data to not save.
I have a very reliable friend who finished the study, so I went to look for her data but only found a file with an empty participant field and the rest of the study data is also empty. She used Chrome (which should be fine), but I have no clue why her data did not get saved properly. Posting here to see if anyone can provide some clue—thanks in advance!