Simultaneous Updates on the Shelf Records

Online experiments
, ,
1

Hi all,

I will create an experiment where tens of participants will join at the same time. This experiment will utilize Pavlovia’s shelf records as its database to inform experimentation in future time points (this is a longitidunal study). We will store participants’ performance, counterbalance conditions, and other important data in these shelves so that the design would work well across time.

My worry is that shelf records may be overwritten while “getting” and “setting” values because many participants will join and interact with this database at the same time. There is no real way of testing this potential problem before experimentation because that would cost a lot to do.

Could shelf records handle this traffic and NOT overwrite the database if:
• I “get” records from shelf,
• Change them at the end of each trial,
• Update the shelf database with “set” function?

Code version of this would be something like:

// trial ends
rt_array.push(trial_rt) // we add this trial's reaction time to a vector 
// for example: rt_array = [592, 869, 1233] this is the reaction time vector, we add new rt at the end of each trial

// then we want this reaction time array to be stored in the database for future studies
// for that, we get participants' information from database 
database = await psychoJS.shelf.getDictionaryFieldValue({key: ["study_database"], fieldName: "PARTICIPANTID HERE", defaultValue: "none"})

// change the database variable ONLY locally (e.g. not in shelf)
database["reactionTime"] = rt_array 

// change the database in shelf records
psychoJS.shelf.setDictionaryFieldValue({key: ["study_database"], fieldName: "PARTICIPANTID HERE", fieldValue : database })

The code above should execute quite fast, however, given that we interact with the database via internet we may get some delays.Is it “possible” that these delays can cause problems (overwriting, not receiving the final version of the database) when multiple participants update the database simultaneously? OR are there countermeasures embedded in the shelf system that orders these change requests and execute them one by one?

I would appreciate any support on this, thank you!

2

Hi…I wonder how did you manage to take care of it eventually. I’m about to launch a similar crowded online experiment using shelf to assign different condition files. It kind of worries me if many participants conduct it simultaneously

3

The shelf is not really designed for large number of participants trying to access it every trial. If you want to store data then the best option is to use a dictionary of dictionaries (with the participant ID as the key value), then get records at the beginning of the experiment and edit the record at the end, with no shelf interaction in between.

If you need trial by trial interaction between participants then this could work, but only for small numbers of simultaneous participants.

4

Hey @Xuanyu,

I decided to update shelves after each trial which took a couple minutes per participant to complete. We conducted a readability experiment (reading pasages, answering comprehension and familiarity questions so a trial takes a long time for each participant). We did not have any problems regarding shelf records or related algorithms. Participants that left halfway through managed to continue from the last trial they completed thanks to shelves.

We ran the experiment for 20 to 40 participants simultaneously per session though, not sure what would it look like if it was around 200 participants for instance.

@wakecarter Thanks for the reply! I used dictionary of dictionaries back when I posted this as well, each p got their own dict and that linked other shelves to this dict etc. It worked out good.

Thanks.