Creating and saving personalized output files

Coding
1

Dear all,

I’m planing a study which will probably include hundreds of participants, so I’d need to program an output file (or output files) very carefully thought and with the exact data I need organized in rows and columns.

In order to do that, I inserted a code component in the builder which I named “Data_collector”. In the “begin routine” tab:

output = [["Response", "Time", "Position"]]

in the “each frame tab”:

keys = event.getKeys(timeStamped=trialClock)
for key in keys:
    if key[0] == 'right':
        key_out = "right"
        Press_pos = str(copy.deepcopy(Subject_position))
    elif key[0] == 'left':
        key_out = "left"
        Press_pos = str(copy.deepcopy(Subject_position))
    elif key[0] == 'up':
        key_out = "up"
        Press_pos = str(copy.deepcopy(Subject_position))
    elif key[0] =='down':
        key_out = "down"
    elif key[0] == 'b' and Subject_position != [-475, -225] and Subject_position != [475, 225]:
        key_out = "b"
        Press_pos = str(copy.deepcopy(Subject_position))
    elif key[0] == 'r' and Subject_position == [475, 225]:
        key_out = "r"
        Press_pos = str(copy.deepcopy(Subject_position))
    else:
        key_out = "wrong key"
        Press_pos = str(copy.deepcopy(Subject_position))

    output.append([key_out, key[1], Press_pos])

and in the “end routine”:

np.savetxt(filename, output, fmt="%s", delimiter="\t")

This is still a trial with three of the variables I’ll need to collect and this worked to save files which I can import to excel. However, after inserting these commands, the experiment is running much slower, sometimes some seems to occur some lagging and frame drops (not sure) and, for sure, not all key presses are being recorded. I’m guessing this might have to do with “for loop” I inserted, which might be causing some frames dropping, but also not sure. I’m new in programming…

Would you know whether I’m going to a good direction in order to create my output files? How could I do this without the “slowness / lagging / dropping” issue? Is there another, maybe better way to achieve what I need?

Thanks in advance.

Best,

Felipe

Obs - this is related to the same experiment about which I had already created a topic before - “Creating barriers in a grid” -, regarding a problem which was very efficiently and nicely solved by @jderrfuss (Thanks)

2

Hi Felipe,

Why don’t you just use the TrialHandler / ExperimentHandler classes to do all of your data saving for you? You can easily add custom information to the output file via thisExp.addData('variable_name', variable_value), where thisExp is the name of your ExperimentHandler

The for loop itself is a very simple one (although it could be greatly streamlined): it is unlikely to be causing a performance problem even when running on every screen refresh. I suspect your timing problems are due to the use of copy.deepcopy(). There are actually very few times you need this command, so I’m sure it could be avoided here. e.g. try streamlining things like this:

key = event.getKeys(timeStamped=trialClock)[0]
if key in ['right', 'left', 'up', 'down']:
    key_out = key
elif key == 'b' and Subject_position != [-475, -225] and Subject_position != [475, 225]:
    key_out = 'b'
elif key == 'r' and Subject_position == [475, -225]:
    key_out = 'r'
else:
    key_out = 'wrong_key'

# get the position. Not conditional, so just have code here, once.
# scalar values, so no need for deep copying:
press_pos_x = Subject_position[0] 
press_pos_y = Subject_position[1]

# save data:
thisExp.addData('key_out', key_out)
thisExp.addData('press_pos_x', press_pos_x)
thisExp.addData('press_pos_y', press_pos_y)

Hopefully you can see the advantage to removing the duplication. e.g in your original code, the position isn’t saved when 'down' is pressed. Maybe that was intentional, but if not, that sort of thing creeps in easily when the same code is repeated.

Now my code above is assuming that you only have one key press per trial. If not, later ones would over-write the earlier one, so it would need to be adjusted.

3

Thanks so much, Michael.
Actually, participants will move a circle across a grid using the arrows and press “r” (6x in average) in a specific position to produce rewards and “b” (6x in average) to build barriers though which the circle can pass. Therefore, they will be allowed to press these keys many times each trial and in need to record all presses (including the “wrong” ones).
That’s why I used the deep copy, so all sequential presses are added to the list without overwriting the previous presses. Is that correct? I did the same to build barriers… See the experiment attached.
Thanks!
Felipe

Felipe_Corchs_exp.psyexp (13.5 KB)

Grid
Grid1473×829 71.3 KB

4

But deep copying can be very slow and in your code above would be the most likely candidate for causing the timing disruptions too are experiencing. You don’t need to go that route if you just access the individual elements of the coordinates: as scalars, they don’t require copying. Deep copying is a rarely-used feature of Python code and there is usually a better approach to be had.

What you could do is maintain lists of your key_out and press_pos_x and press_pos_y values (i.e. set those lists as empty at the start of the trial, then append new values to each list during the Every frame tab code). In the End routine tab code, you would then save the values. Given that you have a reasonably small number of values per trial, I’d recommend saving each to its own column in the data file (e.g. key_out_0, key_out_1, etc). It doesn’t matter that some trials will have differing numbers of presses. The extra columns will just be blank for those trials.

The alternative is to save the entire list of each variable into a single column, but that makes the analysis process much harder.

5

Michael, thanks so much.
Splitting the press_poss in x and y worked to save the entire sequence without over-wrighting the earlier positions (see attached). Nice! Is that because they are new names of variables?
Unfortunately, the performance is still disrupted when getting the keys responses (but not when I exclude this). Curiously, if I insert a keyboard component in builder asking to store “all keys” the problem with the performance occurs as well… Please, note another deep copy in “barriers”, but when this component is on, but “Data_collection” is not, the performance is ok.
Another idea about what can be causing my problem?
Thanks!

Experiment_Felipe_Corchs.psyexp (10.8 KB)

Grid
Grid1473×829 71.3 KB

6

Hi Michael. Sorry about resuming this question, but I ended up having to register the relevant variables every x secs (probably 0.1 sec) instead of when the participant pressed a key (and, therefore, the routine was finished). Is there a way to use Trial/experiment handles like that?
Best,
Felipe