Searching for a file in the working directory

AsiaShuster · November 20, 2020, 9:28pm

Hey!
I’m creating an online experiment, where each participant would get their own unique conditions file. I’ve set it up using tips I found here in the forum, by dynamically naming the conditions file, based on the Participants’ ID.
I’d like to add a piece of code that would search for the .csv before reaching the loop, and in case it can’t find it it would load a default .csv file, that I will put there (just in case I mess up something, I like to have a contingency, so as to not crash the whole experiment).
I’m new to python and to javascript, but in matlab I’d do something like:

conditions_filename = [‘s’ participant_id ‘.csv’];
default_file = ‘default_stim.csv’;
try
load(conditions_filename)
catch
load(default_file)
end

or, alternatively,

if ~isempty(dir(conditions_filename ))
conditions_filename = default_file

Any idea how I can achieve that with psychopy and pavlovia (so without importing any libraries)?
Thank you!

thomas_pronk · November 27, 2020, 10:54am

Hey!

It’s possible, but it’s a bit tricky to accomplish; you would need to add quite a lot of JavaScript to have it work robustly. I’ve got a simpler suggestion that might work too though: put all the participant-specific conditions in a single conditions file and select the right rows depending on the participant. Add your fallback rows in there too.

Best, Thomas

AsiaShuster · November 30, 2020, 2:38pm

That’s a great suggestion, I’ll give it a go. Thanks!

AsiaShuster · November 30, 2020, 7:37pm

Hey Thomas,
Thanks again for your help!
If I understand correctly, this solution would require me to upload the csv ahead of the loop (because i will need to find the relevant row index, based on the participant ID, to then give to the loop as the “selected row”)?

so based on that assumption, and browsing through some posts on this forum for uploading a csv file without using panda, I wrote this code:

# load the csv file
cond_file = 'all_messages.csv' # name of csv file, where each row is aimed at a single particiapnt + last row a fallback option
sID = str(expInfo['participant'])
trial_data = data.TrialHandler(trialList= data.importConditions(cond_file), nReps = 10, method='sequential', extraInfo= expInfo) # import the csv file

# find the subject-specific row in the csv file
if sID in trial_data[:]['receiver_id']: # if the subject id matches one of the items in "recevier id" column of the csv
    row_in_file = trial_data[index(sID)]['receiver_id'] # the row to select for the loop is this index
else: # if the subject id can't be found
    row_in_file = len(trial_data[:]['receiver_id'] ) # pick last row as a default

this is very much not working, I’m not even getting an error message, so no idea how to go about debugging this. As I’ve mentioned, I’m a total beginner and don’t understand what TrialHandler is doing exactly, so am probably using it wrong. As a result, I am also unsure about how to reference a column in the conditions file (e.g., if I wanted to print out the 2nd row in the “receiver_id” column, how would I do that?)

Some more info:
-I learned that I need to change the JS code for the trial_data bit, which I have, probably unsuccessfully.
-Unfortunately, my subject ids are a combination of strings and digits, so I can’t just use them as indices.

help

thomas_pronk · December 1, 2020, 12:47pm

Help is on it’s way!

This one is indeed a bit tricky, since JS doesn’t support the kind of vector operations that Python with numpy does. Tomorrow I’ll build a little prototype to illustrate a nice approach.

thomas_pronk · December 3, 2020, 11:41am

Hey again! Before writing an example, a question. Does trial_data contain the conditions data in a nice JSON structure? Could you print trial_data to the browser console and share a bit of the output you get then?

If so, I’ll provide a code snippet that shows how to parse it (using some example data structure for simplicity)

AsiaShuster · December 3, 2020, 6:29pm

Sure!
Is this what you meant?:


psychopy.data.TrialHandler(
    data={'ran': masked_array(
  data=[[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
        [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
  mask=[[False, False, False, False, False, False, False, False, False,
         False],
        [False, False, False, False, False, False, False, False, False,
         False],
        [False, False, False, False, False, False, False, False, False,
         False]],
  fill_value=1e+20,
  dtype=float32), 'order': masked_array(
  data=[[--, --, --, --, --, --, --, --, --, --],
        [--, --, --, --, --, --, --, --, --, --],
        [--, --, --, --, --, --, --, --, --, --]],
  mask=[[ True,  True,  True,  True,  True,  True,  True,  True,  True,
          True],
        [ True,  True,  True,  True,  True,  True,  True,  True,  True,
          True],
        [ True,  True,  True,  True,  True,  True,  True,  True,  True,
          True]],
  fill_value=1e+20,
  dtype=float32)}
    autoLog=True
    extraInfo=<class 'dict'>
    finished=False
    method=sequential
    nRemaining=30
    nReps=10
    nTotal=30
    name=
    origin=<class 'str'>
    originPath=<class 'str'>
    seed=None
    sequenceIndices=<class 'numpy.ndarray'>
    thisIndex=0
    thisN=-1
    thisRepN=0
    thisTrial=[]
    thisTrialN=-1
    trialList=<class 'list'>
)

I’ve also printed trialList and this is what it looks like:


[OrderedDict([('self_lie', 15), ('self_truth', 10), ('other_lie', 3), ('other_truth', 9), ('resp', 0), ('response_SOLO_keys', 2), ('door_truth_SOLO', 2), ('door_lie_SOLO', 3), ('door_empty1_SOLO', 1), ('door_empty2_SOLO', 4), ('sender_id', '0'), ('receiver_id', 'onl')]), OrderedDict([('self_lie', 25), ('self_truth', 11), ('other_lie', 12), ('other_truth', 37), ('resp', 99), ('response_SOLO_keys', 2), ('door_truth_SOLO', 3), ('door_lie_SOLO', 4), ('door_empty1_SOLO', 2), ('door_empty2_SOLO', 1), ('sender_id', 'onl'), ('receiver_id', '999')]), OrderedDict([('self_lie', 20), ('self_truth', 10), ('other_lie', 6), ('other_truth', 17), ('resp', 0), ('response_SOLO_keys', 2), ('door_truth_SOLO', 2), ('door_lie_SOLO', 1), ('door_empty1_SOLO', 3), ('door_empty2_SOLO', 4), ('sender_id', 'def'), ('receiver_id', 'def')])]

which is essentially my csv file (so looks promising!) though I don’t know how to search within it (as a reminder, I want to search within the column ‘receiver_id’ for a cell equal to the participant’s id as it appears in expInfo)

thomas_pronk · December 4, 2020, 12:47pm

The printed trialList looks promising indeed! I’ll first ask a couple more question, so that I’m sure we are on the same page. Below “is this what you meant” you provided some python script. Are you building your experiment via builder that generated this python script?

Meanwhile, could you share your experiment with me? My username is “tpronk”. Here are instruction on how to do that: https://www.psychopy.org/online/fromBuilder.html#posting-the-issue-on-the-forum

AsiaShuster · December 4, 2020, 3:58pm

Hey Thomas!
I’ve added you to the project on Git (it’s called ASH_receivers, and the relevant part of the code is under // Initialize components for Routine “welcome”).
I have been working on this on Builder (+code snippets) and haven’t attempted yet to upload to pavlovia/git (until now), because I first tried to make it work on Psychopy, and got stuck there. In any case, to generate the abovementioned code I added print(trial_data) or print(trialList) to the code snippet in the Builder version.
Please let me know if I can provide more information!

thomas_pronk · December 7, 2020, 3:47pm

Got it! Should I examine ASH_vReceivers or ASH_vReceivers_vTest?

AsiaShuster · December 7, 2020, 5:10pm

ASH_vReceivers! Sorry for not being clear.

thomas_pronk · December 8, 2020, 6:33pm

Check! I tried some stuff out, but I didn’t get anything elegant to work for dynamically selecting rows in your conditions file. I’ll ask around a bit to see if I missed anything.

AsiaShuster · December 10, 2020, 5:38pm

i’ll keep my fingers crossed! i can’t think of an alternative way to run my experiment with the large n that i require, so i really hope you can help me :-/

thomas_pronk · December 11, 2020, 11:29am

Hey! Just a little update: the meeting I’d like to ask this in is delayed until next week, but I’ll be sure to bring it up!

thomas_pronk · December 24, 2020, 2:45pm

Took a while, sorry for that! I do have a nice solution now. See this demo: https://gitlab.pavlovia.org/tpronk/demo_dynamic_loops/

Also pinging @wakecarter:

This demo is a (hopefully :)) simplified version of one of yours; maybe a nice candidate for the Crib Sheet?
To finish an ongoing loop, use snapshot.finished instead of trials.finished

Cheers T

AsiaShuster · January 5, 2021, 7:39pm

Hi Thomas!
Sorry for taking a while to get back to you and thank you for writing this awesome demo!
The online link you’ve shared ran smoothly, but it’ll be easier for me to edit the psychopy version and merge it into my own, than to write in the JS code on Git (I’m more fluent in Python than JS…). So I tried downloading the files from Git and running the psyexp file locally. I’m afraid that didn’t work (I got past the participant ID screen, but the task wouldn’t load). No error message output to share, unfortunately. Any idea why that could have happened? I’m using Psychopy3 v2020.2.5.

Cheers,
Asia

thomas_pronk · January 5, 2021, 8:48pm

Definitely :). This demo doesn’t work in PsychoPy yet. I’ll ask the Python team how to elegantly implement this in PsychoPy and get back to you

Topic		Replies	Views
Creating participant unique csv files, detailing trial conditions Online experiments csv	2	874	April 23, 2020
conditionsFile as a parameter Online experiments	3	313	October 5, 2020
Select conditions file based on the participant's response Builder	5	819	September 1, 2020
Use a variable for the conditions file path on pavlovia Online experiments	1	984	December 9, 2021
Loop through folder? How to find conditions file only with string Online experiments	3	324	July 24, 2020

Searching for a file in the working directory

Related Topics