Difference in egi.ms_localtime between computers?

[Jin_Jeon]

Hello,

I’ve been struggling to figure out using PsychoPy with PyNetstation module to connect with EGI’s Netstation. I figured out the past problems, and am encountered with one question with that could be possibly linked with synchronization issue.

Issue that I am having is that the egi.ms_localtime() shown on PsychoPy computer is significantly different from the TCP/IP log’s egi.ms_localtime(). See screenshots of the last three events sent:

This is last three custom events of ‘beep’ that I named, shown on the PsychoPy computer. The local time of “Beep sent at 585301043” should be reflected in below TCP/IP log’s last item.

However, at the very bottom item, ‘beep’ here the local time is 91770.

The times here are really off with 585301043 vs. 91770, like the units are totally different. I used both egi.ms_localtime() when sending the time, and made sure that system clock for both machines are very similar (off by around 4 ms). So I am wondering what the problem is.

Below is the really basic python code that I used for this.

#################################################################
#																#
# 		Barebone code for sending signal to Netstation 			#
#																#
#################################################################

from __future__ import absolute_import, division
import os  # handy system and path functions
import sys  # to get file system encoding
import time
import logging

import egi.simple as egi 
ms_localtime = egi.ms_localtime     
ns = egi.Netstation()
ns.connect('10.0.0.42', 55513)
ns.BeginSession()
ns.StartRecording()
ns.sync()

def beep(): 
	ns.sync()
	log_time=egi.ms_localtime()
	print "Beep logged at " + str(log_time)
	ns.sync()
	sent_time=egi.ms_localtime()
	ns.send_event(key='beep', timestamp=egi.ms_localtime(), description='below is sample table', table={'abcd' : 1, 'efgh' : 2}, pad=False)
	print "Beep sent at " + str(sent_time)
	diff = log_time - sent_time
	return diff

Thank you! @imnotamember

[imnotamember]

This is due to computer clocks being different. While this is a topic a really love as I’ve spent far too much time investigating it, I’ll save you the trouble and say plainly–computer time is different between every computer. The differences stem from lots of things:

1. Your computer manufacturer (Dell vs. Mac vs. HP)
2. Your computer’s generation (Macbook Air 2012 vs. Macbook Air 2013)
3. Your computer’s hardware (https://www.engineersgarage.com/mygarage/how-computer-clock-works)
4. Your computer’s manufacturer’s age of same generation (Apple generally okay with this, definitely not on PC’s though; if two laptops of the same model of the same year were manufactured a few weeks apart, they could contain different IC’s and that could change things)
5. The alignment of the sun, the moon, and the stars (I have yet to be proven wrong here )

To get around these issues, timing is typically done in a relative manner with respect to experimental design. That’s what you’re probably seeing here. Think of it as the difference in time-zones. If I’m in Central Standard Time in North America and your Greenwich Mean Time in Europe, we can experience 1 hour of time as the same duration. Even though for me the time duration would be from 12:00PM - 13:00PM, for you it would be 17:00PM-18:00PM.

The calls to ns.sync() are essentially calls to NetStation to say, “Our clocks are currently 124392 milliseconds apart right now, so let’s agree that all timestamps after now assume this time difference.” That way, regardless of if the Mac says 3:00AM and the other computer says it’s 18:17PM, they are agreeing on their difference in time and working relative to that difference.

To highlight this, just check your timestamps in NetStation. Run this code and let me know if your timestamps are accurate regardless of being part of NetStation’s or PsychoPy’s log.

#################################################################
#																#
# 		Barebone code for sending signal to Netstation 					#
#																#
#################################################################
import time
import egi.simple as egi 
ms_localtime = egi.ms_localtime     
ns = egi.Netstation()
ns.connect('10.0.0.42', 55513)
ns.BeginSession()
ns.StartRecording()
ns.sync()

for index in xrange(10):
	# Delay our loop iterations by 1 seconds.
	time.sleep(2) # In the future, use PsychoPy's `core.wait(2)` for more accurate time delays
	# Create our `log_time` and send it to NetStation
	ns.sync()
	log_time=egi.ms_localtime()
	print "Beep logged at " + str(log_time)
	# Don't use `timestamp=egi.ms_localtime()` in the future, it's really inaccurate time-wise. Instead just make `timestamp=None` as I've shown below.
	ns.send_event(key='evt1', timestamp=None, description='below is sample table', table={'abcd' : 1, 'efgh' : 2, 'logT': log_time, 'sntT': 'Not available yet', 'indx': index}, pad=False)

	# Delay our events by 2 seconds.
	time.sleep(2) # In the future, use PsychoPy's `core.wait(2)` for more accurate time delays

	# Create our `sent_time` and send it to NetStation
	sent_time=egi.ms_localtime()
	# Don't use `timestamp=egi.ms_localtime()` in the future, it's really inaccurate time-wise. Instead just make `timestamp=None` as I've shown below.
	ns.send_event(key='evt1', timestamp=None, description='below is sample table', table={'abcd' : 1, 'efgh' : 2, 'logT': log_time, 'sntT': sent_time, 'indx': index}, pad=False)
	print "Beep sent at " + str(sent_time)
	# This should be around 2 seconds
	diff = log_time - sent_time
	print index, diff

Then go to NetStation, check your logs. You should get 2 events. Calculate your timestamp differences in NetStation like so:
Difference in events = evt2 - evt1

If those times don’t come out to be 2 seconds different (keeping in mind your NetStation values are in milliseconds I believe) then there’s something much more nefarious at play here.

Hope that helps you and others who get stuck on timing issues. Let me know if things don’t work or don’t make sense.

Best,
Josh

[Jin_Jeon]

Hey Josh @imnotamember,

Thanks for the reply. I also had a similar code that automatically sends events in fixed time interval+increments to test what you suggested here. I’m posting the screenshots below for the results from your code:
First is the log from Python, and the second is from Netstation’s TCP/IP Log.

The results were around 2 seconds (2000ms) with +/- 6 ms. I’m assuming this is not much of a problem? I’m assuming the small millisecond difference comes from implementing the code. And those differences displayed on NetStation’s TCP/IP log were identical with mostly 6ms (Calculated from table values: 4.sntT - 3.logT).

I was curious though while the logT and sntT are displayed identical to the Python’s log, I wasn’t sure what was printing out those ‘23177731’ timestamps on Netstation’s TCP/IP log. And I was sensing somehow this could be adversely affecting the timing issue that you’ve been talking before.

However, so far if I were to neglect that weird ‘23177731’ value, the only difference seems to be within 10ms and the event keys are also reflected timely in the waveform display.

Best,
Jin Jeon

[imnotamember]

Sorry, should have been clearer. The time I want you to notice is the odd time. That’s the time in NetStation for when the timestamps are received. The units and quantities don’t matter, just that they are precise within your eeg file in relation to the start of your eeg file.

Below is an image highlighting what numbers you should care about. Subtract the second loop event from the first and you’ll get ~2seconds.

Make sense?

[Jin_Jeon]

@imnotamember,

Oh yes, that makes better sense. Just to clarify a bit more, are you saying that what matters is the time difference of 23179739-23177731=2008 (which is ~2seconds)? This would mean that there is only about 8ms difference in terms of time sync, and that it is close enough?

I wanted to clarify on your note on “units and quantities.” So having the unit difference of 608390361 and 23179639 isn’t much significant as long as the time difference is being kept in sync?

Thanks so much!

[imnotamember]

Precisely, your time in NetStation time (in this scenario 2xxxxxxx) is the critical value. That’s the time saved in your eeg file for when the event occurred.

Yes, your numbers could be 3749598377475850827 in PsychoPy and 3 in NetStation. So long as the subsequent events show an equal difference it doesn’t matter. The only caveat there is that you want to make sure your NetStation times are stored properly in your eeg file after you end a recording. Test your scripts heavily before collecting real data.

On a side note, the added benefit of using this system is that you can specify times that are different from your time of sending events. E.g.- if you wanted to send an event after a response was made such that your timestamp ends up at the start of your previous trial, you just need to provide your time from when the previous trial started. That’s the real beauty of this system over parallel port triggers (which are great for precision, but not for information). Although 999 out of 1000 people use PyNetstation as a simple triggering system so don’t worry if that’s your intention.

[Jin_Jeon]

Thanks for the quick and detailed clarification. Yes, it makes totally sense. I was frustrated if such value difference would potentially cause any problem. I am currently re-writing the experiment code from simple form, and testing it gradually, so hopefully it’s all good.

On note of the benefit, I do realize that in the past, the organization used ports instead of TCP/IP and couldn’t get much about the information. Now that we can get more info with PyNetstation, I think we can now attempt try flexibly doing more analysis on NetStation itself too. Thanks for the help again!

[imnotamember]

Gladly! I love supporting the Open Source/Open Science community. While I’m not always available to respond quickly, I’m happy to contribute what I can.

While this problem is considered resolved, I do want to make a special note here that parallel port/serial port triggering is an infinitely more precise manner of sending flags. What you gain with TCP/IP flags is the benefit of information which is easier to organize and utilize when processing your data. There are going to be issues with “offsets” as you’ve noticed. Your timestamps need to reflect time based on screen refreshes and not on clock time. You’ll also need to get a measure of your eeg’s timing “offset” to test how large your time fluctuations (jitter) might be. If you want to read more on the topic, there was an interesting discussion about it here. Timing offsets are not trivial. If you were recording EEG at 100hz and you have variance of +/-8ms, no big deal since your timing precision is less than your recording frequency (1 data point every 10ms). Change that to recording EEG at 1000hz and variance of +/-8ms, you’re flag is failing on 15 samples of data. This might not mean much if you’re interest is a large & slow component like the P300b or similar, but if you’re studying a P50, you’re going to spread your waveform thin and have pretty inaccurate data on your hands. If you need more info let me know and we’ll work through it.

Best of luck; keep the questions coming!
Josh

[luca.filippin]

Hi,
I’d like to add some considerations to the above discussion. The fact that the difference between 2 Netstation’s event timestamps reflects precisely the difference between 2 timestamps taken by the sender just before those events were sent doesn’t tell us anything about the communication delay, apart from the fact that it is constant. If you want to relate timestamps taken within the machine running your experiment to timestamps taken within the Netstation, you need to know this delay. One way to estimate it is to calculate the RTT (Round Trip Time) of a sync:

before = egi.ms_localtime()
ns.sync()
after = egi.ms_localtime()
RTT = after - before
delay = RTT / 2
...
# Now send an event with the localtimestamp +  the information about the delay
ns.send_event(key='MyEvent', timestamp=egi.ms_localtime(), description='test', table={'dely': delay}, pad=False)

In order to take into account the clock offset between the 2 machines, you should sync “frequently enough”. That being said, even this method doesn’t ensure the best precision. In my opinion, if your Netstation provides an NTP server, synchronization via NTP is the way to go. Even so, consider that much depends also on the kind of events you run in your experiment and on the accuracy and precision you want to address: for example, if you play a sound and want to send an EGI event that corresponds exactly to the very moment (within 2 ms) in which that sound is heard (or comes out of the speakers), then you should know that recording a timestamp just before the play instruction in your code almost surely doesn’t give you that exact time. In fact, there’s always a hidden delay due to the processing chain (sound library, OS, sound driver…) to take in account, and this can be far from being negligible (even decades or hundreds of ms).