Experiment data loss and disparity between number of sessions, aborted sessions and completed sessions

[metaresearch]

jsPsych version 7.1.2:

I’m reaching out to share and seek advice on some significant issues we’ve encountered regarding data storage on GitLab servers for our Pavlovia jsPsych experiments. Despite having the “Save incomplete results” feature turned off, we are noticing discrepancies in data storage across several experiments. I’m detailing the issues below in hopes of gaining insights or solutions from the community.

1. Experiment: Int_study1

• Pavlovia Data: 40 sessions; Coins - Assigned: 63, Reserved: 11 (fluctuating), Consumed: 1; Various states of completion.
• GitLab: No data.

2. Experiment: Int_study2

• Pavlovia Data: 22 sessions; Coins - Assigned: 65, Reserved: 6 (fluctuating), Consumed: 1; 1 completed, 6 aborted.
• GitLab: Only one data file, created manually.

3. Experiment: Int_study3

• Pavlovia Data: 35 sessions; Coins - Assigned: 60, Reserved: 12 (fluctuating), Consumed: 1; No completions, 6 aborted.
• GitLab: No data.

4. Experiment: Int_study4

• Pavlovia Data: 29 sessions; Coins - Assigned: 59, Reserved: 10, Consumed: 2; 2 completed, 4 aborted.
• GitLab: Only the repository with 2 regularly stored data sets by participants.

Has anyone else experienced similar issues? Any insights into why these discrepancies are occurring or how to resolve them would be greatly appreciated.

Additionally, if there is a way to recover the missing data or ensure more consistent data storage, please share your advice or suggestions.

Thank you all for your time and help.

[wakecarter]

I don’t use or know anything about jsPsych beyond what I’ve seen on this forum, but here are my thoughts.

1. Are you using the jsPsych-pavlovia plugin? If so, what version?

Is data automatically saved with PsychoJs upon subject completion? How to check this before running study?

I think I sorted out my issue: I hadn’t used the jsPsych-pavlovia plugin.

Does this platform support jsPsych v7?

Hello @Josuke , Since jsPsych changed their plugin system for version 7, we have a new, specific pavlovia plugin for it. It is available on the pavlovia.org server at this URL: https://pavlovia.org/lib/jspsych-7-pavlovia-2022.1.1.js Version 7.1.2 of jsPsych is also now available on the server at this URL: https://pavlovia.org/lib/vendors/jspsych-7.1.2 I have also prepared an illustrative experiment, that showcases how to use the new plugin. You will find it here: Alain Pitiot / jsPsych7_simp…

2. Are you saving using CSV or database?

Jspsych-pavlovia plugin 3.0.0 error saving data

So I thought I had changed the save format from DATABASE back to CSV. I guess it’s not super clear which option is selected and which is deselected on the experiment page in Pavlovia. Changing the save format to CSV solved the problem. You can double check which option you’re using by looking at the Saving Format column of the Dashboard. I’ll keep this here in case anyone else has the same issue.

3. Have you defined any variables twice?

Credits not being used, and data not saving, for "Running" mode in jsPsych?

Hi Katie, OK, this looks like the problem is that var timeline = [] is defined twice, and the second time is after the initialisation of pavlovia_init, which means that pavlovia_init is cleared from your timeline before the task. To fix the problem I removed the second initialisation of var timeline = [] on line 36 of your code. Hope this helps, Becca

4. Have you checked the positions of pavlovia_init and pavlovia_finish?

Data not saving to Pavlovia's servers (jsPsych)

Hi Andreja, I’m pleased to report that I’ve just fixed the issue. I moved the pavlovia_init and pavlovia_finish into the experiment Timeline, but in different places than I had previously tried. Thanks so much for your help and alerting me to this issue. Ella

5. Does your experiment try to do something else (like launch another experiment) while saving data?

Pavlovia not saving data for >20% of participants - specific to one program. [jsPsych]

I am glad to read that the issue has not occurred again, at least not yet, with the 60s pause. I do very much believe that the loss of data you experienced had to do with the asynchronous data saving calls being interrupted, as mentioned earlier. I have now reviewed the plugin situation. Alas, I don’t think there is much I can do: jsPsych does not have a callback mechanism I could use to tell it to wait for data to be saved (incidentally, I believe that also applies to their own approach to sav…

[metaresearch]

Response to Suggestions and Insights on Pavlovia jsPsych Experiment Issues

Thank you for your valuable input on the challenges we’ve been facing with our Pavlovia jsPsych experiments. After careful review of the points you mentioned, here are the updates and considerations based on our current setup:

Use of jsPsych-pavlovia Plugin

• Status: Implemented
• Details: We are utilizing the jsPsych-pavlovia plugin, specifically lib/jspsych-7-pavlovia-2022.1.1.js, which is compatible with our jsPsych version (7.1.2).

Data Saving Method (CSV or Database)

• Status: Implemented
• Details: We use CSV only.

Variable Definitions

• Status: Checked and Confirmed
• Details: Our code does not suffer from the issue of redefining the variable var timeline = []. It appears to be defined correctly without duplication.
  The only variable which is defined multiple times is the local variable curr_progress_bar_value to update the progress bar.

 on_finish: function () {
        var curr_progress_bar_value = jsPsych.getProgressBarCompleted();
        jsPsych.setProgressBar(curr_progress_bar_value + update);
      },

Could this still cause the problem? This is the way how the update of the progress bar by the jsPsych community.

Placement of pavlovia_init and pavlovia_finish

• Status: Correctly Implemented
• Details: The pavlovia_init command is correctly placed at the start of our timeline, and pavlovia_finish is at the end, adhering to the recommended practices.

Interruption During Data Saving

• Status: No Interruption
• Details: Our experiment’s code does not include operations that could interrupt the data saving process, such as triggering another experiment.

Any additional advice or insights from the community would be greatly appreciated.