JsPsych Saving Options on Pavlovia

Online experiments jsPsych
1

URL of experiment: Jeffrey Chen / mfmb-task-shared · GitLab

Description of the problem:
So I know that the JsPsych-Pavlovia plugin automatically stores participants’ data as a CSV file in your GitLab directory according to JsPsych’s native data structures. This is convenient but the formatting can get messy.

Are we able to code experiments in JsPsych and save our participant data to CSV with custom formatting? Or are we beholden to how JsPsych formats data from trials. I’ve tried running the scripts included in this reference: “Storing Data Permanently as a File” (Redirecting), but received a 405 server error in the Google Chrome debugger console. I haven’t been able to diagnose whether this is due to Pavlovia denying the write/send request (and only allowing data collection through the JsPsych-Pavlovia script) or something else.

Related question - Does Pavlovia host MySQL databases? My lab would prefer to store the data to a MySQL table on Pavlovia instead of locally as a CSV. If not, what are some options that have worked for others looking to save to MySQL?

Thank you all!

2

I was just wondering if you ever received an answer to this question. I have also been wondering about making changes to the format of the data file before it gets saved to Pavlovia.

Personally, I would be happy enough to have a csv file for each participant, but I would like to be able to change the formatting of the csv file so that it only stores the information that I want it to store and in the format that I would like it stored rather than just saving the entire jsPsych.data file as a csv.

It would be good to know what kind of flexibility there is regarding the data file that gets saved on Pavlovia.

3

I’m not very knowledgeable of jsPsych, but I’ve got a question/suggestion. Can you specify what to store from within jsPsych perhaps?

4

Hi Thomas,

Thanks for your response.

To be honest, I have not even managed to upload my jsPsych script to Pavlovia yet (this is a steep learning curve for me!). However, my understanding is that Pavlovia just generates a csv of the entire jsPsych.data file (which stores everything) and saves that somewhere on Pavlovia and/or GitHub.

The problem is, I don’t want everything saved in that format because it causes knock-on issues later when I’m trying to import my data into R. I can set up the data how I want it to be formatted, and generate a csv file in jsPsych, but I don’t think I can just save this instead of the entire data set.

It would be good to get some clarity on this - if I can just save my own variables (data frames) instead of (or as well as) the entire data file, it woudl make my life much easier.

I also still need to get my script hosted on Pavlovia, but this also causing me a bit of a headache… though that is probably another question for another thread!

Many thanks,

Cai

5

Hey again!

I read up a bit on how jsPsych + Pavlovia works. There is this a Pavlovia plugin that takes care of interacting with the Pavlovia server. When it stores the data, it retrieves this data by calling this function
jsPsych.data.get().csv()

You give this command as follows:

var pavlovia_finish = {
  type: "pavlovia",
  command: "finish",
  participantId: "JSPSYCH-LONGITUDINAL-2019-20"
};
timeline.push(pavlovia_finish);

Now I’m brainstorming a bit on how to make this suit your needs; not sure if we could easily implement the suggestion below, but just exploring what might work for you. Imagine we add a “preprocessor” function to that command that receives jsPsych.data.get().csv() and returns a filtered version of this csv, which is what gets stored on Pavlovia.

filterFunction = function (csv) {
  // Filter out stuff
  return csv;
}

Would that help?

6

Hi Thomas,

Thanks again for your input.

I’m no expert on this, but I don’t think that will work.

jsPsych.data.get().csv() just takes the whole data set and then converts it into a csv file. That is, it stores the entire data set as a csv file. I have been playing around with it a bit and I can’t see how to edit the original data file (jsPsych.data), or rather I can’t seem to delete data from it (I can only add data to it).

As far as I can tell, the filter funtion only works on a temporary file rather than the original jsPsych.data file. You can use it to filter out data to create your own data structure (indeed, I have done this to create my own csv file that I ask jsPsych to store locally), but I can’t see how to eventually tell Pavlovia to store my new file instead of (or as well as) the original data file.

However, if you are asking me whether it would be helpful to include such a pre-processing stage in the Pavlovia plugin code… well, then that would be a resounding YES from me! It would mean that I could edit the data file to better suit my needs before it gets stored on the Pavlovia server.

7

What would be even better, is if I could create a data structure and then Pavlovia woudl let me save that as a csv instead of the jsPsych.data structure.

If that is at all possible, it would make my life a million times easier.

8

The approach I pitch above might also be used for your ideal scenario; just ignore the csv argument passed to the filter function and have it return whatever you’d like. I’ll go pitch this idea now. Meanwhile, some more brainstorming…

We could have function return a CSV or a JSON, though it could be a bit confusing what extension to store the file in. Maybe just keep the CSV extension and have researchers import that as a JSON later on.

9

Thanks Thomas,

Personally, I had a lot of trouble getting R to read the JSON file and even more trouble trying to manipulate into a workable form for analysis. I would be happy enough with just a csv option and my guess is that most researchers would agree.

If I could just tell Pavlovia what data to save it woudl be amazing!

Cai

10

Okeydokey. I’m discussing it with my senior dev, update soon!

12

I built it, now waiting for a code review, so that I’m sure it’s solid. Meanwhile a suggestion. I noticed that jsPsych already offers all kinds of data filtering options. You could try them out in the browser console while the task is running. If my modifications pass the review, you’ll have access to these options.

13

Thank you!

14

Done! Here you can find a modified version of the Pavlovia plugin that allows you filter out data and/or upload arbitrary data to the Pavlovia server. It comes with a demo task and a bit of documentation. I hope this suits your needs!

15

That’s great - thanks Thomas :slight_smile:

1 Like
16

Hi Thomas,

Thank you a lot for this function ! I tried many hours to do that because I wanted to save JSON data but I’m not good enough ! You give an answer to my question on this post, now it’s simple :

    dataFilter: function(data) {
      let myjsondata = jsPsych.data.get().json();
      return myjsondata ;
  }

That way the file keep the csv extension but the text inside is json in fact. How could I put my post to resolve and make a link to this post ?

17

Happy to read this post solved it! About your question on using JSON as an extension instead of CSV, consider changing this line in the plugin.

18

Hi Thomas,

thanks for your answer concerning the json extension, I used it and it worked well. But i have one question, it seems that this part of the code is used only when the experiment finished without any problem. When there is incomplete data the data are not json nor csv, just some text with coma separator. Is it possible to fix that ?

19

Hey @F_Elisabeth, I guess so, but I’ll need to do a deep dive for it. I put in on my 2do list, but it might take a couple of days before I’ve got a result for you.

1 Like
20

@F_Elisabeth, I took a peek at the code. I see that our pavlovia plugin does attempt to store the data when a participant closes their web-browser half-way through a task, but that isn’t very reliable; it uses features that web-browsers have been disabling because they can be abused by malicious web-sites. Is this the scenario you’re thinking of when you speak of incomplete data or is it perhaps something else I’m not considering?

21

Hi, it’s okay ! It was just in case of a crash during the experiment or something like that. I have some pretest such as an “audiogram like” trial so in the case of a crash it would be interesting to not doing again but I’m conscious we have to do with what the browsers let us do. Anyway, thank you to had a look at it !

1 Like