Running Experiments on Own Server

Online experiments
1

Hi all,

I am looking for resources that would help me setup and run experiments online on our own server.

I have gone through many posts but couldn’t find good resources or get specific answer on whether it is possible or not.

Please do share any resources or links that could help me.

2

A short answer would be: no, I’m afraid it’s not yet possible.

A slightly longer answer: we’re working towards making this possible, but it could take a while, since the current architecture of PsychoJS is very intwined with Pavlovia.

3

Hi @thomas_pronk,

Thank you for your response.
Is it possible to get a rough time frame when this would be possible ?

4

Hey Dhruv,

Well, I can give you a time frame of when we would know. Until December, the team is focusing on bug-fixing and automated testing procedures (I mean PsychoJS. In December, we’d like to make a roadmap for which new features we’d like to implement when. Deploying on other servers is a feature quite frequently requested, so I hope it’s going to be high on the list.

Best, Thomas

5

I just wanted to follow up and ask if you were able to complete the roadmap in December as you had anticipated, @thomas_pronk. We are anxiously waiting to deploy PsychoPy experiments to our own web server and would appreciate any news on this matter. Many thanks in advance!

6

The shorter answer is still that it hasn’t been implemented. We’ve had other priorities I’m afraid.

The question I always ask to users that request this is what the purpose is? Namely, are you asking in order to handle your institution placing restrictions on you, or are you asking because you don’t want to pay the fees supporting Pavlovia? The longer answer depends on what the purpose is of the independent server.

7

IRBs and privacy boards seem to (rightfully or wrongfully) consider self-hosting to be the gold standard for collecting and storing participant data. We self-host all other systems that come into contact with participant data (e.g., survey software, booking software, research database software, etc.), which makes seeking approval for a study smoother and more predictable for our researchers. It also gives participants peace of mind if we can tell them that their data will only be stored on our own servers and not shared with third parties.

Last summer, there was a post by you or someone on your team stating that you would work on a locally hosted solution in Aug/Sep that year. So it sounded like enabling users to self-host PsychoPy experiments was not only planned but also on the short-term roadmap. That whole thread (https://discourse.psychopy.org/t/hosting-online-studies-on-our-own-server-not-pavlovia/15203/), however, was deleted at some point, suggesting there may have been a change of mind on this matter, but this was not communicated publicly.

The predominant way open-source server software works is that you can choose between (1) just downloading the software and hosting it yourself, and (2) paying for the convenience to have the software hosted by the people who created the software, so you don’t have to deal with all the complexities of running your own web server. We are very used to choosing option (1) in that scenario, and I think I had erroneously applied that model to Pavlovia (and a hypothetical self-hosted “light” version of Pavlovia).

I think the long time period of not knowing whether we (in the EU) would be allowed to use Pavlovia after the end of the Brexit transition period (on Dec 31, 2020) also created a visceral sense in me that it would be critically important for us to be able to self-host PsychoPy experiments. Fortunately, with the Pavlovia server having moved to France three weeks ago, this worry is now unfounded.

In summary, we don’t absolutely need to host experiments locally (anymore), but we were looking forward to all the benefits that this scenario would provide. Half a year ago, it looked like a self-hosted solution getting implemented was relatively imminent, but that thread (and maybe others?) was deleted and communication on this matter seems to have gotten more vague in the meantime. Since this is a quite frequently requested feature, I was hoping you could give us slightly more insight into your general thoughts on this feature and on whether it just seems like your thinking has changed or whether it actually did.

Thank you for all your work.

2 Likes
8

There are posts going back at least 2yrs which contain the same statement: “We have other priorities”. Not only the fire worried me but more importantly: local legal restrictions require some institutions to use local servers. Institutions can either pay a lot of money to create their own server infrastructure (because of regulations) or pay a site license. That’s fine. But currently it looks like there is no option → Open source psychopy online means → “pay for Pavlovia”. We as researchers and research group leaders usually do not have the power to overrule our local authorities. Even when there is grant money, it could be a privacy policy that restricts the usage of an external server…

9

As previous posts have discussed, there are two possible issues at play here:

  1. one is whether you need to self-host for privacy/security/legal reasons
  2. the other is whether you want to self host to avoid funding the continued development of PsychoPy/JS

First, let me explain why this has taken a while, we are a very small team (that’s why we charge so little) and we have to make choices about what we spend our time on. Should we prioritise working on adding Microphone Component, or fixing a browser-compatibility bug in Movie presentation, or should we prioritise creating satellite servers? The general answer is that I’d rather we spend our timing developing the code.

That said, for those where there is a genuine privacy/security/policy issue (i.e. issue #1) we have been working on a solution where we work with your IT team to install a mini-pavlovia to receive data. It’s only available to those with site-licenses, at least in the first instance, as it will take a fair bit of support and we don’t have capacity to roll that out and support it for individuals. If you want to be part of the beta-testing on that then let us know.

3 Likes
10

Dear psychopy developers

Could you shed some more light on the future plans regarding this issue. Are self host online experiments coming down the pipeline - or is it just off the table?

I think this is a very important issue, since online data collection probably will be the preferred use case for a lot of users. It certainly is for me.

Given the prominence given to the fact that the project is open source and free, I was actually surprised to learn that this doesnt apply to the code that makes the experiments run online.

There may be good reasons for this. In the answer above Jon alludes to monetary reasons. It is notoriously hard to create sustainable income model for open source development. It would be completely understandable if the code for ‘online use’ is retained as a closed source, pay-for-use addendum to the project (albeit I think it would greatly diminish the usefulness of the project … the open source benefits, no vendor lock-in, etc., also applies to the online part).

This is just speculation. In any case, whatever your plans are regarding this issue, I hope you can share them in a straight-forward, transparent manner. Thank you :slight_smile:

11

The update here is that, for users wanting their own local server to get around IRB/security requirements, we can now provide you with a local-install option. E-mail us at sales@opensciencetools.org if that’s something you need. At the present time we can only support that for institutions with site licenses - the amount of support time in getting it set up is substantial enough not to be sustainable for individual users.

I believe we have been pretty open and explicit about our need for revenue. The code for the JavaScript library (PsychoJS) is fully open source. Only the hosting service is closed source and provides our sustainability model. PsychoJS could not have been developed to this level on evenings and weekends by volunteers, so the question of whether our model is “useful” should take into account whether you want the package to do the things it does.

I believe the key advantages of open source are all maintained, despite the closed-source hosting service:

  • Open Science: you can inspect every part of the code that controls the way your experiment runs for participants - that’s all JavaScript and fully available. There are no black-boxes where you have to hope we did it right. The hosting service is the only part that’s closed-source and that isn’t needed from an open science perspective because it doesn’t impact the way your study runs
  • Extensibility: You can adapt and extend PsychoJS/PsychoPy as needed and contribute your changes back to the project(s) on github if they’re useful more widely
  • No lock-in: Because you have access to all PsychoJS code, if you want to save data to your own server you can adapt the code to do that, assuming you have the technical skills to write it. Note that degree of lock-in is the same for any open-source project - if you create your study in OpenSesame it only runs in OpenSesame unless you have the technical skills to adapt OpenSesame (which is possible because the code is available).
  • Low cost: I think, even with Pavlovia fees, this is about the lowest-cost option available (because we’re only looking to cover our modest costs, not to make profit). It’s certainly the cheapest option to include hosting. If you use a self-hosting option (e.g. Lab.js or jsPsych with a Jatos server) then the staff time and server costs would very likely be more than the cost of Pavlovia credits/license

It’s true that we’re offering an unusual model that sits between the expensive commercial outfits and the no-fees open source options, and I personally think that’s a very useful option for you to have. But if you prefer the self-host option there several alternatives out there for you.

hope that helps to make the position more clear
Jon

3 Likes
12

I’m teaching a programming for psych course, and found this thread when looking for a way for to self-host so students could test run their experiments online with their peers.

Digging in, I came up with two solutions:

Basically, I did what Jon suggests:

Because you have access to all PsychoJS code, if you want to save data to your own server you can adapt the code to do that, assuming you have the technical skills to write it.

Hope this is helpful to anyone else looking for self-host options for prototypes/tests or if you have research restrictions on using third-party servers. This is in no way trying to detract from the Pavlovia service - it’s a convenient solution for many scenarios and a great way to support the hard work of the PsychoPy developers.

1 Like