Alpha testing announcement: Bayesian Adaptive Design for decision making experiments

Development
1

Dear all

What is this?
I’m developing a package for PsychoPy which will allow you to run Bayesian Adaptive Design in your experiments. Think staircases, but adaptive, Bayesian and optimal. The closest thing to this is QUEST+.

The first experimental domain we are applying this to is Delayed And Risky Choice (DARC). So we are talking delay discounting (aka inter-temporal choice), risky choice, and combined delayed and risky choice tasks.

We originally implemented this in Matlab, and the work was done by myself (University of Dundee) and Tom Rainforth (University of Oxford). We have a pre-print here https://psyarxiv.com/yehjb but this is currently undergoing major revision into two separate papers. As part of that effort I am porting the code from Matlab to Python, and embedding it within PsychoPy.

What does Ben want?
At this point I am looking for some alpha testers. I am primarily after 4 things at this point:

  1. Anyone: Let’s work through any download/installation/running problems.
  2. Decision making researchers: Is this going to be useful to you? What more would you like to see?
  3. PsychoPy users/developers: Comments are welcome on the clarity of how I’ve structured the PsychoPy experiment in the builder view + various short code snippets.
  4. Experienced Python folk: This is my first Python package. Any tips on improving the overall code, or best practice tips would be greatly appreciated.

How do I help?
Go to the GitHub repo and either clone it or download the .zip file (just look for the big green download button). The repo is here https://github.com/drbenvincent/darc-experiments-python. The install instructions are at the bottom of that page, as are the Python/PsychoPy version requirements. Let’s see if they are clear enough to follow :slight_smile: As the instructions say, I’ve got a PsychoPy demo there (/psychopy/demo/experiment.psyexp) which has a nice GUI to help you explore the various experiment types and models available.

If you have comments/feedback/problems, then feel free to either write here in this thread, or to create a GitHub Issue.

NOTE 1: This is pre-release code and I am making changes to it very frequently. Improvements are still being made, and this is not recommended for actual data collection at this point. I hope to get to a research ready state soon :slight_smile: For the GitHub users among you - I am just making commits to the master branch, so you can update at any point to get the latest code. No need to worry about branches at this point.

NOTE 2: The Beta testing phase should be much more stable and reliable. This stage will focus on feedback about usability, features, cognitive models which would be most useful for decision making researchers.

2

Hey there,

Fantastic and awesome code. It works wonders on PsychoPy! Is there any version out there that works on Pavlovia with JavaScript code?

Thanks,
Phil

3

Hi Phil

Thanks for the feedback :slight_smile: I have to say while the code is out there, we’ve been forced to put this on hold while we get through other important tasks. So that’s clearing the deck of other papers and also surviving the move to online teaching. But we hope to properly release this code with more formal announcements and examples at some point!

We don’t have any immediate plans to implement it in JavaScript, but we obviously recognise how awesome this would be. I think in the medium term, we’d either have to bring a JavaScript expert on board to port it over, or wait until we can write a grant to pay someone to do it.

My more immediate priority (at some point when I get time) is to port some simpler adaptive methods across. Do you know if staircases have been implemented in Pavlovia/JavaScript yet?

4

Those other important tasks are certainly not small undertakings! Thanks for the swift reply. I am unaware of any staircases implemented in Pavlovia/JavaScript yet, though I will let you know if I find any in my search!

5

I think @richard is still making noises about implementing existing staircases in JavaScript. There is also the possibility of support from the professional JavaScript programmers now working on PsychoJS.

1 Like