Is it possible to get the exact time of a flip in psychoJS

I would like to get the exact time of a flip in psychoJS.

In psychopy this can be achieved by win.getTimeOnFlip().

Is it possible for me to edit the psychoJS code and add in an equivalent function of win.getTimeOnFlip() for javascript?

2 Likes

Hi rkempner,

I’m afraid this isn’t possible yet. W3C has proposed an API for it, but no so far, no browsers have implemented it. https://developer.mozilla.org/en-US/docs/Web/API/Frame_Timing_API/Using_the_Frame_Timing_API

(shameless plug alert). If you’d like to know more about ways to approximate the timing of a frame flip in a web browser, see the online supplement to my paper. TLDR: it doesn’t really matter which method you use. https://osf.io/4fq3s/wiki/home/

Best, Thomas

2 Likes

Thanks for this.

In the psychopy python code, win.TimeOnFlip() is implemented by getting the time in the trial right when we are done waiting for the next flip. I’m curious why we can not follow the same logic for editing the psychoJS source code and make the following edit, here is my pseudocode on top of the Window’s render() function psychoJS source code under the “POSSIBLE NEW ADDITIONS” comment:

		// blocks execution until the rendering is fully done:
		if (this._waitBlanking)
		{
			this._renderer.gl.finish();
		}
	}

           # POSSIBLE NEW ADDITIONS
           get the time in the routine right when the screen is 
      done rendering and add that to some variable for future access

	// call the callOnFlip functions and remove them:
	for (let callback of this._flipCallbacks)
	{
		callback['function'](...callback['arguments']);
	}
	this._flipCallbacks = [];

	// log:
	this._writeLogOnFlip();

	// prepare the scene for the next animation frame:
	this._refresh();
}

It would be great if it’s possible, but web browsers don’t provide that information yet. They might if the W3C API I listed above would be implemented. Until then, the best we can do is approximate it via other timestamps that are available, which I cover in the other link above.

I might have some good news too, but first a question: imagine we could obtain it, what would you like to do with this timestamp?

Re: what would you like to do with this timestamp?, see below

I am working on a serial reaction time task where we want the next target to be presented some consistent interval after the previous target is responded to.

Although having a consistent interval does not seem possible because the reaction time can happen in between screen refreshes, we would like to know the interval from a response to the onset of the next target presentation, and that interval will be important in our analysis. Computing that interval requires knowing approx the exact time of screen flips.

Thanks! For the accuracy of this interval (from response N to stimulus N + 1) there are two factors in play:

  1. The delay between the moment you request stimulus N + 1 to be presented and when it’s actually presented (the flip). This tends to be about 10 ms, but in worst-case scenarios it can be up to 50 ms (though this is uncommon)…
  2. How accurate the RT is measured. In this accuracy you can distinguish a couple of parameters, such as a mean lag variation between measurements. Both parameters tend to vary between devices (operating systems and web-browsers).

There are a couple of peer-reviewed papers on this, with the standard PsychoPy paper being
“The Timing Mega Study”. This paper offers some summary measures of the factors above.

In my own paper (“Mental Chronometry in the Pocket”) I also plot distributions and assess how the accuracy of RT measurements would affect the power and reliability of different designs. For experimental designs the inaccuracy isn’t much of a problem and for individual difference designs only in specific cases.

I hadn’t considered a design like yours though. Indeed, accuracy affects the results your manipulation, but it could be averaged out given sufficient trials. If you can tell me more about your design (and perhaps prior studies you’re replicating) I could make a rough assessment.

Thanks for pointing out these two factors. Yes, I will tell you more about our design, thanks for offering to help.

We are implementing a serial reaction time experiment with several targets. We are investigating how the reaction times to these various targets varies as a function of some manipulations. Participants will press corresponding keys when these targets are presented. During each trial, which is implemented as one routine, the target is presented immediately on the first flip of that trial and the keyboard component collecting responses is set to be synchronized with the screen so the keyboard starting point resets on the first flip of the routine. Then, during this routine participants have about 500 ms to respond, which is specified in frames for consistency. If participants responds before that 500 ms deadline, then we want this routine to end after about 500 ms after that response. If, on the other hand, the participant misses the deadline, we want this routine to end after about 500 ms after the deadline, so that the entire trial is 1 second. Then, since this routine is in a loop for the number N trials we have in our SRTT, the onset of the next target is immediately after the previous iteration of the routine ends. Having reliable reaction times is our top priority. Here are the most critical times we need to control: the keyboard component needs to be synchronized precisely with the onset of the target presentation, and we would like for the interval between response on trial i and onset of the target on trial (i+1) to be consistent to control for any effect of different preparation time contributing the reaction time. At least, we would like to be able to record the interval between response on trial i and onset of the target on trial (i+1)

Also: I want to understand a bit more about what web browsers lack that our local computer have which enable the psychopy python version to implement a timeOnFlip() function.

Looking at the psychopy python implementation for getting the time on the flip, it appears that this is approximated by checking the time in the experiment right after the script is done waiting for the next flip. The value of self._frameTime is assigned to a variable to record the time of the flip:

        # waitBlanking
        if self.waitBlanking and flipThisFrame:
            GL.glBegin(GL.GL_POINTS)
            GL.glColor4f(0, 0, 0, 0)
            if sys.platform == 'win32' and self.glVendor.startswith('ati'):
                pass
            else:
                # this corrupts text rendering on win with some ATI cards :-(
                GL.glVertex2i(10, 10)
            GL.glEnd()
            GL.glFinish()

        # get timestamp
        self._frameTime = now = logging.defaultClock.getTime()
        self._frameTimes.append(self._frameTime)

What is special about the web browsers that make it not possible to follow the same logic to approximate the time of the flip? Is there no psychoJS clock that I can use to get the time?

About web-browsers
I’ll make this a “general introduction into web-browsers”. Computers are organized in layers, where high-level layers use services provided by low-level layers. For example, a PsychoPy app runs on a Python interpreter, which in turn runs on an Operating System (like MacOS or Windows), which in turn runs on yet lower layers. The web-browser adds another layer: PsychoJS runs on a JS interpreter, which runs on a web-browser, which runs on an Operating System.

Web-browsers weren’t originally designed to do very funky things; they were originally for presenting static HTML documents. In time, they got increasingly sophisticated, but there are still some important limitations, that take time to resolve. For instance, by design a web app should be compatible with a much larger variety of devices; a Python app won’t run on an iPhone, but a web app would.

Also, web browsers have a lot more security issues to contend with; a Python app assumes a much higher degree of trust from the user than a web app. Hence, a web app is in a more restrictive “sandbox”, so that it has less direct access to the resources of your computer. Worse, sometimes web browsers open up things which are then exploited in hacks, so they are next closed off again. We saw this for instance in the high resolution timer, which was used in certain exploits, so browsers intentionally made the timer less accurate again. For more info, see this post on Meltdown and Spectre.

Returning to your question. I’m not sure why we can’t obtain the moment of a flip (browser technology is a field in itself), but I can tell you that we cannot. Also, given that the standard I listed above already exists for while, but still hasn’t been implemented, I guess it has to do with security.

If you’d like to know more about how web browsers work, there are some nice refs in my Mental Chronometry in the Pocket paper. In particular, the papers by Garaizar et al. go a lot deeper than I did.

About your design
You can safely assume that the level of accuracy you ask for can only be obtained via PsychoPy and not via PsychoJS. And even via PsychoPy, to be really precise, you’d like to have a computer that is optimized for timing accuracy. At my old department, we even had technicians whose job it was to build and configure such computers, and test them via specialized hardware.

The question that remains is: “what if the timing is less accurate?”. My paper can also serve as an entry point into simulations that can give you an estimate of that. In your case, the question boils down to "what if the interval is 900 ms? Or 1100 ms? With a bit of luck prior studies can give you some information about that, which you can plug into the simulation of my paper or the others that I referred to.

1 Like

Thank you for all this.

I should clarify something important about our design: we are just comparing reaction times within-participants so consistent lag does not matter. Accuracy between participants does not matter so much but a good enough precision within-participants is important.

Reading “The Timing Mega Study”, we felt confident that psychoJS could give us the precision we are looking for in the measuring reaction times since the onset of the target, and we have made a plan to run a large number of studies online to get around the small variance in the precision of the reaction times.

With this in mind, do you think that we can get the precision we are looking for in the reaction times? It is clear from the two factors you discuss in the above post, that the accuracy of that response-stimulus-interval will not be good, and I think we can safely ignore this although it is not ideal.

Additionally: above I described my attempts at approximating the time of the flip by following the same logic as psychopy’s timeOnFlip() function. Would my above description give us any reasonable approximation?

About your design. Do you want to compare participant RTs across two (or more) discrete within-subjects conditions?

About how to approximate the flip most accurately It works a bit differently online (read about requestAnimationFrame to learn more about that). I examined a couple of methods for approximating the page flip in the online appendix of “Mental Chronometry in the Pocket” (I call a page flip a refresh in that appendix), but the take-home message is “any method is as good as the others”, so the timestamp logged by PsychoJS will be just as good as any of your custom solutions.

hey @thomas_pronk, apologies for missing your reply on this thread, and thanks for helping out, awesome to have your advice. I am still trying to ensure that my design is suitable for the web, and it would be very helpful to hear your thoughts.

Regarding our design: we have a probabilistic serial reaction time task (PSRTT) where we want to compare RTs for the different targets in the PSRTT within-subject, but the condition of these targets is not exactly discrete since we are assigning a continuous value which is constantly changing to each of the targets based on the probabilistic structure of the task. The experiment is similar to replicating the following task in a more simplistic web-based setting: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5997338/.

From your paper, “For relative RT, between-device variation is removed due to RTs being subtracted between conditions within participants.” This idea is why we think our task will be okay, since we assume that the variation of the different devices can be treated like a different intercept in a mixed effect model.

1 Like

@thomas_pronk Just want to confirm, do you agree that my design and assumptions are reasonable for web-based experimentation?

I confirm that this design looks like one that’s robust against the kind of RT inaccuracies you could expect to find online :slight_smile:

1 Like

awesome – thanks so much for your help !

Hi Thomas, I noticed in the PsychoJS scheduler code that timestamp is set when request animation frame is executed. Is there a way I can access this timestamp and store it in my csv. datafile?

Hello! Please could you share the line of the source code where the timestamp is you want to store? hopefully we can help more then,

Thanks,
Becca

Hi Becca,

I have included the source code below. I am trying to store lastDelta and timestamp. I previously tried to store these in the each frame tab of a code component, but the variables consistently returned undefined. Suggesting, (according to my very poor javascript knowledge) that these variables are locally defined.

Currently, to get the “timestamp” variable, I am running window.requestAnimationFrame(); in the each frame tab of a code component. This gives me the timestamp I need, but i am not sure how ideal it is to repeat code from the scheduler in a code component.

Scheduler source code:

async start()
	{
		const self = this;
		const update = async (timestamp) =>
		{
			// stop the animation if need be:
			if (self._stopAtNextUpdate)
			{
				self._status = Scheduler.Status.STOPPED;
				return;
			}
			// self._psychoJS.window._writeLogOnFlip();
			// run the next scheduled tasks until a scene render is requested:
			const state = await self._runNextTasks();
			if (state === Scheduler.Event.QUIT)
			{
				self._status = Scheduler.Status.STOPPED;
				return;
			}
			// store frame delta for `Window.getActualFrameRate()`
			const lastTimestamp = self._lastTimestamp === undefined ? timestamp : self._lastTimestamp;
			self._lastDelta = timestamp - lastTimestamp;
			self._lastTimestamp = timestamp;
			// render the scene in the window:
			self._psychoJS.window.render();
			// request a new frame:
			requestAnimationFrame(update);
		};
		// start the animation:
		requestAnimationFrame(update);
	}

1 Like