ioHub datastore could not be loaded: msgpack 'can't serialize'

Other
, ,
1

Hello you all!

I’m having problems with the ioHub demos (for eyetracking), in particular the gcCursor one. I have all the required dependencies installed (pip list file: pipList.csv (1.3 KB)) but it gives me this error:

##### Running: C:\Python27\lib\site-packages\psychopy\demos\coder\iohub\eyetracking\gcCursor\run.py #####
WARNING: ioHub DataStore could not be loaded. DataStore functionality will be disabled. Error: 
<type 'exceptions.ImportError'>
ImportError('DLL load failed: No se encontr\xf3 el proceso especificado.',)
['  File "C:\\Python27\\lib\\site-packages\\psychopy\\iohub\\__init__.py", line 68, in <module>\n    import datastore\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy\\iohub\\datastore\\__init__.py", line 16, in <module>\n    import tables\n',
 '  File "C:\\Python27\\lib\\site-packages\\tables\\__init__.py", line 90, in <module>\n    from .utilsextension import (\n']
WARNING: ioHub DataStore could not be loaded. DataStore functionality will be disabled. Error: 
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\psychopy\iohub\client\__init__.py", line 1157, in _sendToHubServer
    bytes_sent = self.udp_client.sendTo(ioHubMessage)
  File "C:\Python27\lib\site-packages\psychopy\iohub\net.py", line 70, in sendTo
    d=self.pack(data)
  File "msgpack\_packer.pyx", line 231, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3661)
  File "msgpack\_packer.pyx", line 233, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3503)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 228, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3382)
TypeError: can't serialize 2
Warning: TimeoutExpired, Killing ioHub Server process.
<type 'exceptions.TypeError'>
TypeError("can't serialize 2",)
['  File "C:\\Python27\\lib\\site-packages\\psychopy\\iohub\\client\\__init__.py", line 1968, in start\n    result=self.run(*sys_argv)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy\\demos\\coder\\iohub\\eyetracking\\gcCursor\\run.py", line 281, in run\n    self.hub.addRowToConditionVariableTable(trial.values())\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy\\iohub\\client\\__init__.py", line 663, in addRowToConditionVariableTable\n    r=self._sendToHubServer((\'RPC\',\'addRowToConditionVariableTable\',(self.experimentID,self.experimentSessionID,data)))\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy\\iohub\\client\\__init__.py", line 1162, in _sendToHubServer\n    raise e\n']

I’m running this on windows10 x64.

Someome knows hot to fix it?

2

Hi Christian,
I think the source for the problem might be truncated in the Traceback. Could you open a new coder project and just insert

import tables

run that script and post the output?

Best,
Robin

3

Hi Robin,

Thank you for your answer and sorry for the delay!!

I have do that and give me error, searching on the pytables page i found the solution for the problem (install the gnuWin32 libraries and the hdf5 library and add the paths to the set with cmd).

After that i had struggles with cython, that were solved with the installation of minGW and the modification of the distutils config file to ensure the use of this tool in the process of compilation (i’ve installed visual compiler for python too).

Finally (and because of that i haven’t answered) i’m fighting with a new error… can i share it with you?

Thanks!

PD: I’ve documented all the steps that gives me good results in this process, it’s possible to post all of them (and the pip list) to help others with the same problem?

4

Hi Christian,
Sounds like progress :wink:
Sure, please share both, errors and solutions, so future searches of users with related problems may end up here. Maybe share the complete solution for this specific problem here (mark the thread as solved) and open a new thread for the new problem.

5

UPDATE

Robin,

I’ve taken my time to reproduce the error and find a good solution to it. After make a full reinstall of the software and all the packages the answer seems more clear now (and, for my surprise, don’t have relation with the tables package :face_with_raised_eyebrow:).

After the installation of the packages that appear on the official psychoPy page, following the steps that appears on each package page, and using the last version of psychopy from github, it gives this error using the coder demo for eyetracking with iohub (gcCursor):

##### Running: C:\Python27\lib\site-packages\psychopy-1.85.4-py2.7.egg\psychopy\demos\coder\iohub\eyetracking\gcCursor\run.py #####
[('S',)]RPC_RUNTIME_ERROR
----- Called by function ----
handle
----- Exc Type -----------
<type 'exceptions.TypeError'>
----- Exc Value ------------
TypeError('data type not understood',)
----- Exc Traceback ---------
['  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\server.py", line 116, in handle\n    result = funcPtr(*args)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\server.py", line 379, in initializeConditionVariableTable\n    return self.iohub.emrt_file._initializeConditionVariableTable(experiment_id,session_id,output)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\datastore\\__init__.py", line 453, in _initializeConditionVariableTable\n    self._EXP_COND_DTYPE = np.dtype(np_dtype)\n']
-----------------
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\psychopy-1.85.4-py2.7.egg\psychopy\iohub\client\__init__.py", line 1164, in _sendToHubServer
    bytes_sent = self.udp_client.sendTo(ioHubMessage)
  File "C:\Python27\lib\site-packages\psychopy-1.85.4-py2.7.egg\psychopy\iohub\net.py", line 75, in sendTo
    d=self.pack(data)
  File "msgpack\_packer.pyx", line 231, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3661)
  File "msgpack\_packer.pyx", line 233, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3503)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3230)
  File "msgpack\_packer.pyx", line 228, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3382)
TypeError: can't serialize 4
Warning: TimeoutExpired, Killing ioHub Server process.
----- Called by function ----
start
----- Exc Type -----------
<type 'exceptions.TypeError'>
----- Exc Value ------------
TypeError("can't serialize 4",)
----- Exc Traceback ---------
['  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\client\\__init__.py", line 1975, in start\n    result=self.run(*sys_argv)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\demos\\coder\\iohub\\eyetracking\\gcCursor\\run.py", line 286, in run\n    self.hub.addRowToConditionVariableTable(list(trial.values()))\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\client\\__init__.py", line 671, in addRowToConditionVariableTable\n    r=self._sendToHubServer((\'RPC\',\'addRowToConditionVariableTable\',(self.experimentID,self.experimentSessionID,data)))\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\client\\__init__.py", line 1169, in _sendToHubServer\n    raise e\n']
-----------------

That seems to be the same that the one opening this post. The way that i find to solve it was based on the ‘Exc Traceback’ info that implies a relation between the use of msgpack and numpy, searching on pipy page i found a package that solves the error (partially) and can be installed with pip:

pip install msgpack-numpy

The output after this was:

##### Running: C:\Python27\lib\site-packages\psychopy-1.85.4-py2.7.egg\psychopy\demos\coder\iohub\eyetracking\gcCursor\run.py #####
RPC_RUNTIME_ERROR
----- Called by function ----
handle
----- Exc Type -----------
<type 'exceptions.TypeError'>
----- Exc Value ------------
TypeError('data type not understood',)
----- Exc Traceback ---------
['  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\server.py", line 116, in handle\n    result = funcPtr(*args)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\server.py", line 379, in initializeConditionVariableTable\n    return self.iohub.emrt_file._initializeConditionVariableTable(experiment_id,session_id,output)\n',
 '  File "C:\\Python27\\lib\\site-packages\\psychopy-1.85.4-py2.7.egg\\psychopy\\iohub\\datastore\\__init__.py", line 453, in _initializeConditionVariableTable\n    self._EXP_COND_DTYPE = np.dtype(np_dtype)\n']
-----------------
ioHub Server Process Completed With Code:  0

Anyway, the demo execute completely and saves the hdf5 file at the end.

Do you have any clue about how to solve this error?

Thanks!

PD: I have this on python: pipList.csv (1.4 KB)

6

Hi Christian,
numpy does not like whatever is stored in the np_dtype variable. It’s hard for me to track down, since the ioHub module is quite nested and I don’t have an eye tracker, so these lines of code wouldn’t even be reached on my local setup.

What eye tracker do you use?

I’m not sure how familiar you are with Python coding, but maybe we could get a better understanding of what’s happening, when you could put a print statement in "C:\Python27\lib\site-packages\psychopy-1.85.4-py2.7.egg\psychopy\iohub\datastore\__init__.py" just above line 453, like

print(np_dtype)

7

Robin,

Looking into the file i’ve found a debug line with this code just before the line:

print2err('np_dtype: ',np_dtype,' ',type(np_dtype))

uncommenting this i’ve this output:

np_dtype: [('EXPERIMENT_ID', 'i4'), ('SESSION_ID', 'i4'), (u'session_id', [('S',)]), (u'trial_id', [('S',)]), (u'ROW_INDEX', [('S',)]), (u'BLOCK', [('S',)]), (u'IMAGE_NAME', [('S',)]), (u'TRIAL_START', [('f',), ('8',)]), (u'TRIAL_END', [('f',), ('8',)])] <type 'list'>

So numpy don’t like objects with type ‘list’?

PD: I’m using an eyeTribe eyetracker :slight_smile:
PD2: Not familiar with python but with c/c++ and matlab. I want to think that i’m a fast learner so you just have to ask and i’ll find a way!