psychopy.org | Reference | Downloads | Github

Support for interactive studies

#1

Hi,

I wonder if there are any plans to add support for interactive studies? I’m thinking about studies that require communication or coordination between computers (the kind that are common in experimental economics). An example would be a study in which you ran a group of participants at the same time and wanted to display relative performance after each trial.

If there aren’t any such plans:

a) Is this something you would consider adding? (I suppose that there isn’t much demand for this among psychologists.)

b) Do you have any tips if I wanted to code something like that myself?

Best,

Aljaz

#2

Actually I have thought about this a little but nothing has been created directly to support it. I imagine doing live versions of prisoners dilemma etc.

For your use case: if the computers are connected to some common network drive then the easy thing to to is have them all write to a folder there and then have them all read from that folder. If each client:

  • writes a tiny text file with a unique filename and containing the score of the participant.
  • reads all the files there to work out the current distribution of scores

Creating these sorts of things by a graphical interface seems hard to imagine but I’m always open to suggestions if you can envisage how it would look and be reasonably general.

1 Like
#3

Have you experienced other packages that do this well? How did their interface work?

#4

I was thinking about something like that, yes. I suppose that in principle this text files approach could be taken to implement designs of any complexity, no? Or do you see any potential problems?

How to present the same randomised stimuli on two computers?
#5

I haven’t used any such software yet.

#6

Yes, in principle there’s a great deal you can do with simple mechanisms like this. You just need to think carefully about issues like:

  • not having two machines write to the same file at once
  • the order in which things may (or may not happen), such as what happens if one participant gets to the point that they need info from the other participant but the other one hasn’t got there yet
  • don’t try moving large files or writing any more to the shared space than you have to

Basically, there will be wrinkles but you should be able to get a long way with simple read/write of text files

1 Like
#7

I believe the approach I would take for data exchange would involve a simple database, otherwise you almost certainly will run into locking issues. Databases are very easy to access from within Python, and you don’t need to re-invent the wheel.

1 Like
#8

Thanks for the tip. Could you suggest a resource to learn about this stuff? I haven’t done anything with databases yet.

otherwise you almost certainly will run into locking issues

What if each new piece of information is written to a new file, so that no file editing ever takes place? Wouldn’t this be safe?

#9

Hello @Aljaz, I will be very busy today, but I can post some example code tomorrow. :slight_smile:

2 Likes
#10

I had super-busy day today and will have to postpone my example code to tomorrow, sorry!

#11

Just a quick reminder (no pressure, I’m in no hurry).

#12

Thanks! I really totally forgot, too many other urgent things to do! I will get back to you until Sunday evening (CEST)! I have an example somewhere around here, just need to find it…

1 Like
#13

Hi @Aljaz, good news: I found the example code! Bad news, it’s a mess, as the DB stuff is not properly separated from the rest of the experiment (it’s relatively old code, and was never really used). I have started to refactor the whole thing, and will try to add multiprocessing support tomorrow, so you could actually simulate several computers/players participating simultaneously, using only a single machine. Give me another day or two :slight_smile: Thanks!!

1 Like
#14

Or… just use simple text files where machines only write to a file with their name but read from any, and don’t worry about databases. :wink:

#15

Hi Richard,

I’m now working on my first interactive study, so a code example would come very handy. I understand refactoring takes time, so would it be possible to post just an existing, non-working excerpt so that I get an initial idea of how this would look like?

Thanks!

#16

This is some code I developed a while back, with slight modifications to try it out on a single computer. The solution is based on SQLAlchemy, which allows you to map database structures to Python objects (object relational mapping, ORM). It’s currently using the SQLite backend, a file-based, serverless DB system, which is great for playing around with it locally.Do not place an SQLite DB on a fileserver for concurrent access from different computers. Most network file systems do not implement locking correctly, and the DB will break. You may want to use MariaDB instead once you need a “remote” DB.

The example code assumes we have 5 independent players (implemented as Python threads via joblib); it runs 10 trials. On each trial, a random response and RT are generated, as well as a timestamp – independently for all participants. These data are then stored in the database – again, each of the five threads accessing the database “at random” – we do not have to worry about locking, the DB takes care of this!

The database structure is defined in the class ResponseState – that’s our ORM right there. We drop the DB table storing the results every time the script is run via ResponseState.__table__.drop(engine); this is because we use the trial number as a primary key, and it wouldn’t be unique anymore on a second run.

The wait_for_responses() function simply checks whether we have already collected responses from all players in the current trial; once we do, it sets the all_done entry in the database to True, and we can proceed to the next trial.

The simulation can be run by invoking run_experiment(). Please adjust the path in base_dir first.

Once the experiment is over, we can very easily extract the data from the DB using pandas; see extract_results(): It’s basically a one-liner!!!

Please refer to the SQLAlchemy and respective pandas docs for further info.

For local testing with SQLite, I always have my DB open in SQLite Manager, a Firefox addon.

I hope this helps you a bit, it’s really simple, although some things may look a bit weird at first. But once you understand the key concepts, it’s really not difficult anymore.

And a big sorry again for responding so late. Too many commitments, too little time. :disappointed:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import datetime
import sqlalchemy as sa
import pandas as pd
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Column, Integer, DateTime, Float, Boolean
from sqlalchemy.exc import (IntegrityError, InvalidRequestError,
                            OperationalError)
from numpy.random import random_sample, random_integers
from psychopy.core import wait
from joblib import Parallel, delayed


def run_trial(trial_num):
    session = Session()
    response_state = ResponseState(trial=trial_num)
    session.add(response_state)

    try:
        session.commit()
    except (IntegrityError, InvalidRequestError):
        session.rollback()
        session.close()
        raise RuntimeError('Something went wrong here!!')

    # Five "Parallel" "players" (Python threads)
    Parallel(n_jobs=5, backend='threading')(
        delayed(gen_response)(player_num=i) for i in range(1, 5 + 1))


def gen_response(player_num):
    session = Session()
    r = (session
         .query(ResponseState)
         .order_by(ResponseState.trial.desc())
         .first())

    # Create random response as integer in the interval [1, 10].
    response = random_integers(1, 10)

    # Create random RT in the range [1, 3] sec, and wait for the 'response'.
    rt = (3 - 1) * random_sample() + 1
    wait(rt)
    time = datetime.datetime.now()

    r.__setattr__('player_%i_response' % player_num,
                  response)

    r.__setattr__('player_%i_rt' % player_num,
                  rt)

    r.__setattr__('player_%i_time' % player_num,
                  time)

    session.add(r)

    try:
        session.commit()
    except (IntegrityError, InvalidRequestError):
        session.rollback()
        session.close()
        raise RuntimeError('Something went wrong here!!')


def wait_for_responses():
    session = Session()
    latest_trial = (session.query(ResponseState)
                    .order_by(ResponseState.trial.desc())
                    .first())

    player_num = 1
    while player_num <= 5:
        response = latest_trial.__getattribute__('player_%i_response'
                                                 % player_num)

        if response is None:
            wait(0.1)
            continue

        player_num += 1

    latest_trial.all_done = True
    session.add(latest_trial)

    try:
        session.commit()
    except (IntegrityError, InvalidRequestError):
        session.rollback()
        session.close()
        raise RuntimeError('Something went wrong here!!')


def run_experiment():
    trials = range(1, 10 + 1)
    for trial in trials:
        run_trial(trial)
        wait_for_responses()

    print('Experiment completed.')


def extract_results():
    data = pd.read_sql('responses', engine)
    return data


if __name__ == '__main__':
    base_dir = os.path.normpath('D:/Development/Testing/database')
    db_file = os.path.join(base_dir, 'database.db')

    engine = sa.create_engine('sqlite:///' + db_file, echo=True)
    Session = sessionmaker(bind=engine)
    Base = declarative_base()


    class ResponseState(Base):
        __tablename__ = 'responses'

        trial = Column(Integer, primary_key=True)
        all_done = Column(Boolean)

        player_1_response = Column(Integer)
        player_2_response = Column(Integer)
        player_3_response = Column(Integer)
        player_4_response = Column(Integer)
        player_5_response = Column(Integer)

        player_1_rt = Column(Float)
        player_2_rt = Column(Float)
        player_3_rt = Column(Float)
        player_4_rt = Column(Float)
        player_5_rt = Column(Float)

        player_1_time = Column(DateTime)
        player_2_time = Column(DateTime)
        player_3_time = Column(DateTime)
        player_4_time = Column(DateTime)
        player_5_time = Column(DateTime)


    try:
        ResponseState.__table__.drop(engine)
    except OperationalError:
        pass

    Base.metadata.create_all(engine)
1 Like
#17

Thanks Richard! I’m using the txt solution for the current project but will dig into this for the next one.

1 Like