Variables empty in online script (looping over dictionaries)

Blog
1

Description of the problem:
I have an experiment, working in builder, which I am trying to transfer to Pavlovia. I am using importConditions to read in a csv file (see this thread for the backstory), but it seemed from the web console that, though the import line runs, the variable doesn’t exist or is empty.

Relevant portions of the original python code with importConditions:

# ------- import words & distracters -------

testfile = './html/resources/practice_EN_NH.csv'

practList = {}
practListData = importConditions(testfile)
for k in practListData[0]:
    practList[k] = [d[k] for d in practListData]

catchList = {}
for k in practList:
    catchList[k] = practList[k][6:].copy()
    print('catch list ' + k + ':')
    print(catchList[k])
    del practList[k][6:]

Error encountered:

TypeError: "_pj_a is undefined"
    experimentInit https://run.pavlovia.org/Aisa2/dictimport_test/html/dictImport.js:115
    _runNextTasks https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1091
    _runNextTasks https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1094
    update https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1058
core-2020.1.js:1438:12
    onerror https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:1438
FATAL unknown | {} log4javascript.min.js:1:40074
    append https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    doAppend https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    callAppenders https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    log https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    fatal https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    dialog https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:921
    onerror https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:1439

More info: the js code it references for “_pj_a is undefined” is this:

  for (var k, _pj_c = 0, _pj_a = practListData[0], _pj_b = _pj_a.length; (_pj_c < _pj_b); _pj_c += 1) {
      k = _pj_a[_pj_c];
      practList[k] = function () {
      var _pj_d = [], _pj_e = practListData;
      for (var _pj_f = 0, _pj_g = _pj_e.length; (_pj_f < _pj_g); _pj_f += 1) {
          var d = _pj_e[_pj_f];
          _pj_d.push(d[k]);
      }
      return _pj_d;
  }
  .call(this);
  }

so _pj_a is the first entry of practListData.

I am not sure if I have included some code incompatible with psychoJS, or if the problem is coming from elsewhere. Has anyone seen this before?

2

As a test, I tried creating a dictionary directly in my code. This was all right until I tried to access it and create the second variable (catchList). Then that variable seems not to exist. So maybe this has something to do with iterating over dictionaries, and not with importing at all?

Here the web console correctly prints out the variable I manually defined:

{…}
dist1: Array(10) [ "dub", "med", "shove", … ]
dist2: Array(10) [ "pub", "pink", "bank", … ]
list: Array(10) [ 99, 99, 99, … ]
n: Array(10) [ 1, 2, 3, … ]
original: Array(10) [ "sub", "red", "shore", … ]
position: Array(10) [ 1, 2, 3, … ]
​

… but now catchList is not the kind of object I expect; it seems like nothing happened in that loop.

<prototype>: Object { … }
dictImport.js:127:11
practLen: 10 dictImport.js:129:11
{}
​<prototype>: {…}
​​__defineGetter__: function __defineGetter__()
​​__defineSetter__: function __defineSetter__()
​​__lookupGetter__: function __lookupGetter__()
​​__lookupSetter__: function __lookupSetter__()
​​__proto__: 
​​constructor: function Object()
​​hasOwnProperty: function hasOwnProperty()
​​isPrototypeOf: function isPrototypeOf()
propertyIsEnumerable: function propertyIsEnumerable()
​​toLocaleString: function toLocaleString()
​​toString: function toString()
​​valueOf: function valueOf()
​​<get __proto__()>: function __proto__()
​​<set __proto__()>: function __proto__()

Then the error comes when we try to access catchList:

dictImport.js:130:11
TypeError: "catchList.n is undefined"
    experimentInit https://run.pavlovia.org/Aisa2/dictimport_test/html/dictImport.js:131
    _runNextTasks https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1091
    _runNextTasks https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1094
    update https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/util-2020.1.js:1058
core-2020.1.js:1438:12
    onerror https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:1438
FATAL unknown | {} log4javascript.min.js:1:40074
    append https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    doAppend https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    callAppenders https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    log https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    fatal https://cdnjs.cloudflare.com/ajax/libs/log4javascript/1.4.9/log4javascript.min.js:1
    dialog https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:921
    onerror https://run.pavlovia.org/Aisa2/dictimport_test/html/lib/core-2020.1.js:1439
3

@wakecarter this might be important to look into since the method I’m using is now in the crib sheet…

4

I’ve deleted it from the crib sheet for now. I think that it should be possible to create a dictionary instead of an array using my loop method, since one should just be able to append/push the dictionary for each iteration just as I append/push the array values.

5

Hi @wakecarter that’s good for now. From what I can see in your experiment, you’re using the conditions directly in the trial loop. Is there a way to have a bank of stimuli without importing them in a loop?

6

How would you tell PsychoPy what the bank of stimuli are if they aren’t in an Excel file or an array or a dictionary?

7

Hmm, maybe I am misunderstanding. Does TrialHandler import everything before the experiment begins, or do you need to actually step through each trial to access the next bit of data in the excel file?

8

Basically what I need to do is import my csv/excel file take the first ~240 lines for use as “real” words (words that are actually presented to the participant during the learning phase), and then shuffle the other ~160 words for use as “fake” words (ones that were not presented to the participant during learning). The fake ones need to be shuffled and gradually sampled throughout the experiment (without replacement), and the real trials need to be presented only if the participant didn’t already type them during a free recall portion. So I’d like a really flexible way to access the data attached to each of these words.

Maybe I don’t understand TrialHandler enough to do that kind of thing yet, but do you think it would work?

9

I don’t know how to use “TrialHandler” or even what it is apart from a PsychoPy thing that seems to be a code version of a Builder loop.

However, I would use a loop to append all 240 words in a random order to one array the other 160 to a second array. I’d then present the items 240 during the presentation loop and then interleave them with the second array during the test loop (skipping trials where the word has already been recalled).

10

-That’s what I was trying to do with the dictionary above, but maybe I missed something. I’ll try doing it using a builder loop instead of just writing my own loop in code; maybe that will help.

-Unfortunately it’s a little more complex than that; the words are presented in lists of 10, each followed by free recall, then followed by a series of real/fake trials (shuffled together). The number of fake trials changes proportionally to the number of real trials (which depends on the number of recalled words). But I have ways to do all that once my dictionaries are imported.

11

I am still curious why this very simple loop doesn’t work online, though… maybe @dvbridges can shed some light on it?

12

@aisa2, if you are attempting to read a csv or xlsx file online without using Builders loops, I would recommend using a trial handler to handle that for you. Here is the JS code that will work:

myData = new TrialHandler({
		psychoJS: psychoJS,
		nReps: 1, method: TrialHandler.Method.SEQUENTIAL,
		extraInfo: expInfo, originPath: undefined,
		trialList: 'conditions.xlsx',
		seed: undefined, name: 'myData'
	});

trialList = myData.getTrialList()

// Now your data is stored in a list of JS objects
// the equivalent of a list of Python dicts
// To get the value for a particular column and row
trialList[0]['myColumnHeader']  // row 1
trialList[1]['myColumnHeader']  // row 2

// If you want to create two arrays for two types of condition, 
// you can filter the list of objects based on a variable in conditions file 
// e.g., column labelled "condition" for condition 0 and condition 1
cond0 = trialList.filter((trial) => trial['condition']==0)
cond1 = trialList.filter((trial) => trial['condition']==1) 

// Now you can choose to shuffle your new arrays, or leave them sequential
util.shuffle(cond0)
2 Likes
13

@dvbridges thank you so much!
This is the javascript version, correct? Will the python TrialHandler translate correctly using Auto–> JS? And should I use TrialHandler or TrialHandler2?

14

This is JavaScript, so does not require translation. You can put this in the relevant JS tab of your code component (e.g., begin Experiment or Begin Routine). Make sure that you put your conditions file in the resources folder, since PsychoPy will not know you are using them in a code component.

15

I’m interested in code which I can use locally as well. What would I need to put in the Python code for it to mean the same thing?

16

I’m guessing we would:

  • replace the colons in the TrialHandler call with = signs
  • remove the psychoJS: psychoJS line
  • change the method argument to “sequential”
  • change the originPath argument to None instead of undefined

So far my python command looks like:

practListData = data.TrialHandler(trialList= 'practice_EN_NH.csv', 
    nReps = 10,
    method='sequential',
    dataTypes = varNames,   #<-- varNames is just a list of the column names of the file
    extraInfo= expInfo)

Looks like it translates fairly well on the JS side, however I’m still getting this error:

Traceback (most recent call last):
  File ".../dictImport_test_pavlovia/dictImport_lastrun.py", line 90, in <module>
    practListData = data.TrialHandler2(trialList= 'practice_EN_NH.csv', nReps = 10, method='sequential', dataTypes = varNames, extraInfo= expInfo)
  File "/Applications/PsychoPy3.app/Contents/Resources/lib/python3.6/psychopy/data/trial.py", line 854, in __init__
    self.columns = list(trialList[0].keys())
AttributeError: 'str' object has no attribute 'keys'
1 Like
17

@wakecarter it also looks like instead of

trialList= 'somefile.csv',

you have to use

trialList = data.importConditions('somefile.csv')
1 Like
18

Okay, I got it to work! Thank you @dvbridges!

To summarize: use the TrialHandler to import the stimuli. This will need to be edited separately in python and JS; if you are in builder, it is easiest to use a python-only code component and a js-only code component.

You can see a working example here:
https://pavlovia.org/run/Aisa2/dictimport_test/html/

And the code here:

In the python code, the import looks like:

testfile = './html/resources/somefile.csv'
varNames = ['col1','col2','col3','etc']; # names of column headers in original file: this is optional, I think.
stimData = data.TrialHandler2(trialList= data.importConditions(testfile), nReps = 10, method='sequential', extraInfo= expInfo)

In the JS code, it looks like:

testfile = 'somefile.csv'
varNames = ["col1", "col2", "col3", "etc"];
stimData = new TrialHandler({
    psychoJS: psychoJS,
    nReps: 1, method: TrialHandler.Method.SEQUENTIAL,
    extraInfo: expInfo, originPath: undefined,
    trialList: testfile,
    seed: undefined, name: 'stimData'
});

Then you access the file in a row/column fashion, like

stimData[0]['myColumnHeader']

Other notes from @dvbridges on the JS end:

// If you want to create two arrays for two types of condition, 
// you can filter the list of objects based on a variable in conditions file 
// e.g., column labelled "condition" for condition 0 and condition 1
cond0 = stimData.filter((trial) => trial['condition']==0)
cond1 = stimData.filter((trial) => trial['condition']==1) 

// Now you can choose to shuffle your new arrays, or leave them sequential
util.shuffle(cond0)

Bonus note: if you want to select different slices of your newly-imported stimulus set, you will also need to do this in python/JS separately, using [start:end] in python and .slice(start, end) in JS.

19

@wakecarter the equivalent Python code is below, but this will not auto-translate correctly because the JS trialHandler takes different argument values (e.g., see method and trialList args) and the call to the trialHandler parameter trialList (the myData.trialList call) is also different for JS. However, the rest should translate ok :

myData = data.TrialHandler(nReps=1, method='sequential', 
    extraInfo=expInfo, originPath=-1,
    trialList=data.importConditions('conditions.xlsx'),
    seed=None, name='myData')

trialList = myData.trialList
trialList[0]['myColumnHeader'] # row 1
trialList[1]['myColumnHeader'] # row 1

cond0 = [trial for trial in trialList if trial['condition'] == 0]
cond1 = [trial for trial in trialList if trial['condition'] == 1]

shuffle(cond0)
20

Hi Thanks for proposing this and figuring this out. I am new to the code component and have the following questions

  1. what do you mean to better have two separate codes for python and js respectively? Do you mean the code type in the code component? How did you do that? using py->js type and edit respectively?
  2. If js is created and edited, what is the point to maintain a .py file?

Thanks