String encode/decode error

Other
1

Hi,
I’m encountering some issues with installing Psychopy 1.90.1 on both Windows 10 and Windows 7 64bit machines. There seems to be (and this is a guess) a fundamental problem in loading the python libraries. For example, when I try to run the following code

from psychopy import core, event, visual, sound, logging, gui, data
from numpy.random import random, randint, normal, shuffle
import pandas as pd # these are for reading from excel files
import random, os, csv, glob, sys


# Ensure that relative paths start from the same directory as this script
_thisDir = os.path.dirname(os.path.abspath(__file__)).decode(sys.getfilesystemencoding())
os.chdir(_thisDir)

#launches a GUI to get info
expName = u'incomplete letters'
expInfo = {'participant': "",'condition':'1'}
dlg = gui.DlgFromDict(dictionary=expInfo, title=expName)
if dlg.OK == False:
    core.quit()  # user pressed cancel
expInfo['date'] = data.getDateStr()  # add a simple timestamp
expInfo['expName'] = expName

# start a clock for the reaction times
rtimer=core.Clock()# general purpose clock

# Data file name stem for saving data = absolute path + name; later add .psyexp, .csv, .log, etc
filename = _thisDir + os.sep + u'data/%s_%s_%s' % ('P'+expInfo['participant'], 'C'+expInfo['condition'],expName)#, expInfo['date'] # took date out cause it was annoying


# this experiment handler simplifies the saving of data a bit
exp = data.ExperimentHandler(name='incomplete_letters',version='0.1',runtimeInfo=None,originPath=None,savePickle=False,saveWideText=True,dataFileName=filename)


# set up your window
win = visual.Window( units='deg',monitor='office',fullscr=True) #size=(800, 800),
win.setMouseVisible(False)

################################ SET UP STIMULI  ###################################

#read in the whole excel file, you can specify the sheet if you need to
filename1=_thisDir + '\incomplete letters stimuli list.xlsx'
datast = pd.read_excel(filename1)

# build lists with the values from the excel files
images_a = datast['file1'].values
images_b = datast['file2'].values
images_c = datast['file3'].values
images_d = datast['file4'].values
images_e = datast['file5'].values
images_f = datast['file6'].values
images_g = datast['file7'].values
images_h = datast['file8'].values
images_i = datast['file9'].values
images_j = datast['file10'].values
images_k = datast['file11'].values
images_l = datast['file12'].values
images_m = datast['file13'].values
images_n = datast['file14'].values
images_o = datast['file15'].values
images_p = datast['file16'].values
images_q = datast['file17'].values
images_r = datast['file18'].values
images_s = datast['file19'].values
images_t = datast['file20'].values
images_u = datast['file21'].values
images_v = datast['file22'].values
images_w = datast['file23'].values
images_x = datast['file24'].values
images_y = datast['file25'].values
images_z = datast['file26'].values

image_list=[images_a,images_b,images_c,images_d,images_e,images_f,images_g,images_h,images_i,images_j,images_k,images_l,images_m,images_n,images_o,images_p,images_q,images_r,images_s,images_t,images_u,images_v,images_w,images_x,images_y,images_z]

answers_a = datast['answer1'].values
answers_b = datast['answer2'].values
answers_c = datast['answer3'].values
answers_d = datast['answer4'].values
answers_e = datast['answer5'].values
answers_f = datast['answer6'].values
answers_g = datast['answer7'].values
answers_h = datast['answer8'].values
answers_i = datast['answer9'].values
answers_j = datast['answer10'].values
answers_k = datast['answer11'].values
answers_l = datast['answer12'].values
answers_m = datast['answer13'].values
answers_n = datast['answer14'].values
answers_o = datast['answer15'].values
answers_p = datast['answer16'].values
answers_q = datast['answer17'].values
answers_r = datast['answer18'].values
answers_s = datast['answer19'].values
answers_t = datast['answer20'].values
answers_u = datast['answer21'].values
answers_v = datast['answer22'].values
answers_w = datast['answer23'].values
answers_x = datast['answer24'].values
answers_y = datast['answer25'].values
answers_z = datast['answer26'].values


answer_list=[answers_a,answers_b,answers_c,answers_d,answers_e,answers_f,answers_g,answers_h,answers_i,answers_j,answers_k,answers_l,answers_m,answers_n,answers_o,answers_p,answers_q,answers_r,answers_s,answers_t,answers_u,answers_v,answers_w,answers_x,answers_y,answers_z]



################################ INSTRUCTIONS  ###################################

def instpres(string):
    IS=visual.TextStim(win=win, color='black', height=1, text=string, alignHoriz='center',wrapWidth=20)# this is alignment to the screen not the text alignment, irritatingly
    IS.draw()
    win.flip()
    core.wait(3)#prevent accidental presses. 
    responsegiven=False # set the response counter to false
    while not responsegiven:
        for key in event.getKeys(): #accept all key presses
            if  key in (['escape']): # if one of the key presses has been escape then quit
                win.close()
                core.quit()
            elif key in (['space']): # if it matches the response 
                responsegiven=True


instpres('Hello, you will now see a series of images.\n\n\
Please press the space bar until you can identify a letter\n \n\
When you can identify the letter, press the corresponding key on the keyboard\n \n\
Press the spacebar to start a demonstration.')

################################ LISTS AND RANDOMISATION ###################################

# open empty lists for the responses and reaction times
file=[]
keyspressed=[]
reactiontime=[]
letter_id_time=[]
identification=[]
letter_select = range(0,26)# generate a list of numbers bw 1:26
shuffle(letter_select)# shuffle the list

################################ RUN DEMO ###################################

images=image_list[letter_select[23]]# take the image LIST based on the random 1:26 number
answers=answer_list[letter_select[23]]# and the corresponding correct answer

a=0 # simple counter for the image files
continue_loop=True # continue until they get the correct response 
while continue_loop:
    file=images[a]#keep a record for the data file of which image was being used
    image = visual.ImageStim(win, image=images[a])# draw the image [a]
    image.draw()
    win.callOnFlip(rtimer.reset)# reset the clock when the image is presented
    win.flip()
#    rtimer.reset()# set reaction timer to 0
    responsegiven=False # set the response counter to false
    while not responsegiven:
        for key in event.getKeys(): #accept all key presses
            if  key in (['escape']): # if one of the key presses has been escape then quit
                win.close()
                core.quit()
            elif key in (answers[a]): # if it matches the response 
                responsegiven=True # then count it as a response
                keyspressed.append(key)
                rt=rtimer.getTime()
                reactiontime.append(rt)
                identification = 99-a
                continue_loop=False # quit loop because it was correct
            elif key not in (answers[a]) or not (['escape']):#stop accidental key presses by the participant. 
                responsegiven=True # count as a valid response
                keyspressed.append(key)
                rt=rtimer.getTime()
                reactiontime.append(rt)
                continue_loop=True # continue on to the next image because it was incorrect
    a+=1

################################ FINISH DEMO ###################################


################################ START MAIN EXPERIMENT ###################################
instpres('You are now ready to start the experiment.\n\n\
Press the spacebar to start.')

b=0 # create a counter for the letter list

while b < len(letter_select):#for all 26 letters 
    images=image_list[letter_select[b]]# take the image LIST based on the random 1:26 number
    answers=answer_list[letter_select[b]]# and the corresponding correct answer
    letter_start=core.Clock()
    a=0 # simple counter for the image files
    continue_loop=True # continue until they get the correct response 
    while continue_loop:
        core.wait(0.5)# this stops them pressing the space bar too quickly and skipping through too fast
        file=images[a]#keep a record for the data file of which image was being used
        image = visual.ImageStim(win, image=images[a])# draw the image [a]
        image.draw()
        win.callOnFlip(rtimer.reset)# reset the clock when the image is presented
        win.flip()
        responsegiven=False # set the response counter to false
        while not responsegiven:
            for key in event.getKeys(): #accept all key presses
                if  key in (['escape']): # if one of the key presses has been escape then quit
                    win.close()
                    core.quit()
                elif key in (answers[a]): # if it matches the response 
                    responsegiven=True # then count it as a response
                    keyspressed.append(key)
                    rt=rtimer.getTime()
                    reactiontime.append(rt)
                    identification = 99-a
                    continue_loop=False # quit loop because it was correct
                elif key not in (answers[a]) or not (['escape']):#stop accidental key presses by the participant. 
                    responsegiven=True # count as a valid response
                    keyspressed.append(key)
                    rt=rtimer.getTime()
                    reactiontime.append(rt)
                    continue_loop=True # continue on to the next image because it was incorrect
    ################################################## SAVE TRIAL BY TRIAL RESPONSES ##################################
        exp.addData('image file',file)
        exp.addData('key_out', keyspressed[-1])#this takes the most recent key press, without the -1 it just tots them all up, it also magically gets rid of the annoying sq brackets
        exp.addData('reactiontime', reactiontime[-1])
        exp.nextEntry()# this is a critical line, without this it doesn't save the data, it just moves to next line. 
    ################################################## SAVE TRIAL BY TRIAL RESPONSES ##################################
        a+=1
    letter_id_time=letter_start.getTime()
    ################################################## SAVE LOOP BY LOOP RESPONSES ##################################
    exp.addData('image file1',file)
    exp.addData('identification point (%)',identification)
    exp.addData('total letter id time',letter_id_time)
    exp.nextEntry()
    ################################################## SAVE LOOP BY LOOP RESPONSES ##################################
    b+=1

instpres('Thank you, you have finished.\n\n\
Please wait for your next instruction.')

I get the following error

##### Running: X:\Psychology\ResearchProjects\GStothart\BRACE Pilot 2018\paradigm development\incomplete letters pilot\incomplete letters behavioural.py #####
1.5361     INFO     Loaded SoundDevice with PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4
1.5362     INFO     sound is using audioLib: sounddevice
Traceback (most recent call last):
  File "X:\Psychology\ResearchProjects\GStothart\BRACE Pilot 2018\paradigm development\incomplete letters pilot\incomplete letters behavioural.py", line 13, in <module>
    _thisDir = os.path.dirname(os.path.abspath(__file__)).decode(sys.getfilesystemencoding())
AttributeError: 'str' object has no attribute 'decode'

Even if I click on the error, I get another error, which is what is making me think there is something really fundamentally wrong with the install.

Traceback (most recent call last):
  File "C:\Program Files (x86)\PsychoPy2_PY3\lib\site-packages\psychopy\app\coder\coder.py", line 2889, in onURL
    lineNumber = int(tmpLineNumber.split()[0])
ValueError: invalid literal for int() with base 10: '13,'

I’ve spent some time looking into Python version, manual installs, C++ libraries and .net framework version but to be honest I’m a bit out of my depth. Do you have any pointers for things to check?

FYI my old version of Psychopy, v1.85.1, had none of these problems, worked perfectly and if I roll back to 1.85.1 still works perfectly.
Many thanks
George

2

Those issues were both to do with Python3/Python2 issues as we made the code compatible with Python3. I believe they were fixed by the most recent stable release (1.90.3):

3

Great thanks Jon, will try 1.90.3 on Monday. Our institution has made 1.90.1 available as the default managed installation (via a centrally managed software centre app) so sounds like I should tell them to move to 1.90.3.

4

Hi Jon,
I tried installing v1.90.3 using both the StandalonePsychoPy2_PY3-1.90.3-win32 and StandalonePsychoPy2-1.90.3-win32 installers this morning.

Unfortunately I get the same basic error, the error message now looks like:

##### Running: C:\Users\EGI\Documents\My Experiments\George Stothart\psychopy\Fastball pilot for lab\Fastball recall task_Lab_1.py #####
2.4420     INFO     Loaded SoundDevice with PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4
2.4421     INFO     sound is using audioLib: sounddevice
Traceback (most recent call last):
  File "C:\Users\EGI\Documents\My Experiments\George Stothart\psychopy\Fastball pilot for lab\Fastball recall task_Lab_1.py", line 28, in <module>
    _thisDir = os.path.dirname(os.path.abspath(__file__)).decode(sys.getfilesystemencoding())
AttributeError: 'str' object has no attribute 'decode'

Some things have been fixed, e.g. if I click on the error line link in the error message it does now take me to the correct line, but it still looks like there’s some python problem.

Thanks
George

5

Hi @gstothart, the error is found because the string object in Python 3 does not have a decode attribute. In Python 3 str(bytes, encoding, errors) is equivalent to bytes.decode(encoding, errors)(https://docs.python.org/3/library/stdtypes.html#bytes.decode).

The Python 3 PsychoPy should not output the code using the decode function, so what may be happening is that you compiled a script in PsychoPy that uses Python 2. and attempted to run in PsychoPy running with Python 3.

If you are running in Python 3 PsychoPy, you could either recompile the script from builder or replace the line of code causing the error with:

_thisDir = os.path.dirname(os.path.abspath(__file__))  # removing the decode function
6

My apologies, I hadn’t read your error message properly, and hadn’t realised this was a problem with your script rather than with installation. As @dvbridges says, the treatment of unicode and ascii (bytes) has changed with version 3 of Python, and it was in the 1.90 series of PsychoPy that we started handling those differences (hence several “decode” bugs have been fixed). But this looks like a bug in a line of your script (copied form a Builder-output at some time?). That will need you to fix it to be in keeping with the right python version (irrespective of the PsychoPy version).

7

Hi jon and @dvbridges, thanks very much for your replies.
The example code I provided was indeed copied over from Builder when I was first learning to code with Psychopy. This explains why some of my more recent “homemade” code works fine, and the old scripts that contain elements copied from the Builder do not.

However, one other problem I’ve encountered is using the egi Library (C:\Program Files (x86)\PsychoPy2\Lib\site-packages\egi) for integrating an EGI EEG system. This does not use any element of Builder generated code, but has a problem with the socket connection using the following code

import egi.simple as egi 
ms_localtime = egi.ms_localtime     
ns = egi.Netstation()
ns.connect('10.10.10.42', 55513) # sample address and port -- change according to your network settings
ns.BeginSession()     
ns.sync()

I’m sorry I can’t post up the actual error message as I’m at a conference at the moment, but it was along the lines of
AttributeError: 'XXXX object has no attribute ‘socket’

I will post the actual error up next week, but I guess I have two questions in the meantime

  1. Will the EGI add-on be compatible with the new versions of Psychopy? Joshua Zosky (@imnotamember) made an invaluably useful fix for this library last year, perhaps he can advise.

  2. Is there a blog/guide anywhere for relative Python novices that could highlight obvious things to update or check for after the Python2-3 switch?

8

That error suggests you didn’t actually connect to the NetStation computer. Which version of NetStation and which amplifier are your using?

The EGI add-on should be compatible. It doesn’t get into anything complicated as far as class declarations are concerned. I say that with the caveat that I haven’t personally tested it out and one thing that may have changed is the socket module included in Python 2.x versus Python 3.x. For that I’ll need more of the error message to confirm though.

Thanks for your kind words. Glad to know it helps. There was also interest in updating this for the 400 series amps with NTP messaging. If anyone is testing either of those out (python 3.x or NTP messaging) I’d be happy to work on updates in my (limited) spare time.

Python 2 to 3 cheat sheet:
http://python-future.org/compatible_idioms.html

9

Hi Josh, brilliant thanks for your help again! I’ll post up the actual error messages next week, it may be that there’s something very simple to fix now that I know that it is python versions that are the root cause of the problems.
Thanks
George

10

Hi,
so having checked a few different installations I can confirm:

  1. Most of the problems stemmed from old builder derived code not being compatible with new versions of Psychopy, especially those using Python 3.

  2. The EGI plug-in (C:\Program Files (x86)\PsychoPy2\Lib\site-packages\egi) provided with the typical installation and adapted by @imnotamember still works well with the Python 2 version of 1.90.3

  3. The EGI plug-in appears to not be included with the Python 3 version of 1.90.3, @jon - I don’t know if this is intentional because it is not compatible with Python 3 or just an accidental omission.

Thanks again everyone for your help fixing these problems
George

11

Sorry, the egi lib was accidentally omitted on the windows builds. It will be added back for 3.0.0b10 upwards

What was the fix? Is it something that could/should be pushed back up to the GitHub - gaelen/python-egi repo and released? If not is there something I need to do post-install?

12

OK, I’ve taken a look and pynetstation definitely has at least one syntax errors that will prevent it working with Python3 for now.

Using Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 16:07:46) [MSC v.1900 32 bit (Intel)] on win32:

>>> from egi import simple
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\egi\simple.py", line 347
    type(1L): ('long', '=l'),  # '=' -- translation is: four bytes
          ^
SyntaxError: invalid syntax
>>>

1L is no longer valid syntax in Py3.

There’s also a suspicious line if not isinstance(string_key, str): where pynetstation is expecting a string to be returned, but I suspect it will be receiving bytes object from the socket.

I’ve made sure this is still included in the Python2 version of 3.0.0b10 but the adaptation by @imnotamember isn’t something I know about

13

Hey @jon and @gstothart,

Regarding my edited version, that was for clarity, since many colleagues weren’t familiar with python’s function argument handling. All my edit did was provide a function that sets timestamps at the time that the function .send_event() is called to appropriately timestamp events on screen flips. The alternative way to do it with the currently implemented egi module is to just pass None for the timestamp argument.

I’ll gladly make changes to update to Python 3.x if you can give me a minimum version to develop towards.
Best,
Josh

14

Ah, OK, I do have a vague recollection of that issue. People were calling getTime() rather than passing getTime, right? I see so I don’t need to change the packaged lib I guess.

Currently we’re supporting 3.6 as the main target (e.g. for Standalone packages) but we’ll try and continue support for 2.7 as long as it’s not too painful to do so.

15

Yeah, that was essentially the issue. There’s better ways to handle that issue though, which leads me to my next statement: I’ve been contacted recently by a colleague urging me to update the PyNetstation code to utilize a more timing-precise method (NTP instead of TCP). In doing so I’ll update the code for Python 3.6+ and backwards compatibility for Python 2.7+. On a side note, does anyone know the original developer? I’ve been doing maintenance on the codebase and wanted to thank them/ask for permission to take over development as they seem to have moved on.

16

No I managed to interact with gaelen a couple of years ago, who forked it to github, and had also moved on to other things. I never managed to track down the original author that had posted it on Googlecode.