Strange behavior with retina displays + external monitors in 1.90.2 (py2)

[jonathan.kominsky]

Probably related to a previously fixed issue, but I have no idea how:

Retina display windows opening in wrong place in 1.90.0 Py2 version

I’m on a 2017 Macbook Pro with retina display. When I open a window, with or without fullscreening, it centers it on the top-left corner of the screen. This occurs whether I open the window on screen index 0 or 1 (notably, when opening on screen index 1, it puts it in the top left of THAT screen). Everything is on the pyglet backend. By default new windows are opened at 0,0, and it looks like that’s being mapped to the top-left corner now instead of the center of the screen. Notably, it does not…

Basically, I ran the “screensAndWindows” demo with a few permutations after noticing that stuff opens in weird places and at weird sizes on a second screen when the primary monitor is a retina display. It’s far stranger than I expected. I haven’t tried this on the beta yet, so there’s a chance it may have been fixed, but it doesn’t look like window has changed in any relevant way…

There are five strange behaviors:

1. When I put winL on screen 0 and winR on screen 1, both windows appear, but only winL shows the stimuli, while winR remains blank.
2. When I put both windows on screen 1, they report different sizes. winL reports 800, 600 (which would be the expected “size” if it were on a retina display, because of the doubling) and winR reports 400, 300. This is the same as it reports when winL is on screen 0 and winR on screen 1. When both are on screen 0, both report 800, 600.
3. When both windows are on screen 1, while it reports that the two windows should be adjacent (as when both are on screen 0), they are not:
   
   
4. I asked for framerate reports using getMsPerFrame. When both windows are on screen 1, winR reports impossible refresh rates (0-7ms), but inconsistently. This does not happen when both are on screen 0, or when they are on different screens. This is despite the fact that it won’t show anything in winR when they are on separate screens.
5. Here’s where things get super strange. If I put winL (the first window created) on screen 1, and winR (the second window created) on screen 0, winL renders but stays blank, and reports being 800x600, but winR renders as follows and reports being 400x300:
   
   

I’m using a current-gen MacBook Pro running the most recent version of High Sierra (not Mojave) as my screen0, and an old VGA 1280x1024 LCD as screen1, running through an adapter.

The best that I can figure is that there is something about the first versus second window being created and something about pixel scaling that are interacting very badly, and I can’t track down anything in the Window code that would explain this behavior. I’m pretty lost with this one. Anyone have any ideas?

[jon]

I don’t know the answers to these issues off the top of my head. The basic problem is indeed probably to do with retina scaling. The problem we have is that the retina silently rescales things. We’ve added a few places to detect that this has happened and readjust, but you’re probably highlighting some new ones. These look like they might be an interaction with Pyglet and you might find different results using the new ‘glfw’ backend on the PsychoPy 3.0 beta

[jonathan.kominsky]

I’m mostly curious why the order of window creation seems to matter. Do you think that’s a pyglet thing?

[jonathan.kominsky]

OK, so I’ve got some news about this, some good, most confusing. Still on high sierra, now using the most recent version of PsychoPy3 (which is great). @jon you may be interested to hear how glfw is working here.

I finally wrestled glfw into place on my mac. This was nontrivial and not something that I could possibly recommend to anyone who isn’t a power user. Among other things, I discovered that the dylib file that PsychoPy goes looking for is in /opt/local/lib, not /usr/local/lib. Why I could not say. In any case, after updating the glfw module’s init file to include /opt/local/lib in its the list of search paths in _get_library_search_paths() (lines 167-168), “import glfw” in PsychoPy’s python shell now works. Woo! Question: Should I just put in a pull request for that one or is that technically not PsychoPy’s problem?

Then I tried running the screensAndWindows.py demo with glfw backends. This generated new and exciting issues.

Trying to use the multi-screen with no external monitor: Creates only one window, but animates properly. Pyglet, for comparison, just creates both windows as though you’d run the single-screen version. This is pyglet’s usual behavior when confronted with a nonexistent second screen.

Trying to use the single-screen version with no external monitor: Creates two windows but only animates one of them. It looks like it only updates the most recently created window, and while it creates both windows, it does not update or track events for the first one.

Trying to use the multi-screen version with an external monitor connected via HDMI: It fixes the weird behavior I was getting from pyglet! It renders perfectly on the external monitor, right place, right size, right everything. I’ve confirmed that this is glfw doing the heavy lifting, switching it back to pyglet with this very setup it fails like it always did. However…the thing about updating only the most recently created window still applies. It creates a window on the retina display as well, but it’s blank.

So, GLFW may be able to solve the problems I was having, but the big stumbling blocks now are 1) it needs to be much, much easier to install (ideally part of the psychopy installation process) and 2) it needs to be able to update multiple active windows. I tried messing with waitBlanking and it didn’t change anything.

Also, with regard to Pyglet, I have at least a faint sense of why it might be failing so catastrophically. I direct you to line 76 of the cocoa.py canvas file, with the comment that says “FIX ME” in all caps. Apparently macs and multi-monitor support have been shaky in Pyglet for a while, so I’m thinking that it’s defaulting some values to the ‘context’ of the main monitor and that’s driving the weirdness when dealing with multiple monitors with different pixel densities. Unfortunately, I have not the faintest clue how to fix that either.

Any ideas where to direct my efforts from here?

[mdc]

@jonathan.kominsky

So, GLFW may be able to solve the problems I was having, but the big stumbling blocks now are 1) it needs to be much, much easier to install (ideally part of the psychopy installation process) and 2) it needs to be able to update multiple active windows. I tried messing with waitBlanking and it didn’t change anything.

This is something I’m going to look into. I wrote a note about using multiple displays in the source code, it may help you out if you haven’t seen it yet …

    Multi-Display Timing:

        If using multiple displays, waiting for multiple retraces may cause a
        reduction in overall frame rate. Do the following to prevent this:
            1. Pick a display as your primary screen.
            2. When creating a window for your primary screen, set
               swapInterval=1 (default anyways).
            3. Create windows for all other displays with swapInterval=0.
        In most cases, drawing across all displays will be synchronized with
        your primary display. However, there is no guarantee vertical retraces
        occur simultaneously across multiple monitors. Therefore, stimulus onset
        times may differ slightly after 'flip' is called. In some cases visual
        artifacts my arise that affect your data (temporal disparities in a
        haploscope affect perceived depth). If multi-display synchronization is
        absolutely critical, check if your hardware supports 'gen-lock' or some
        other synchronization method.
        Always check inter-display timings empirically (using a photo-diode,
        oscilloscope or some other instrument)!

The GLFW backend has some secret constructor arguments and behaviour that are not yet documented in the API reference. I’m reluctant to do so since testing is a bit thin around those features because it’s all still new. I think the order which you flip the windows counts, you might need to flip the primary display last since it will block until vertical retrace. You may also need to enable context sharing like so:

win1 = Window(...)
win2 = Window(share=win1)

[jonathan.kominsky]

Thanks for this! Share did the trick for the single-monitor version, it now updates both windows. It did not fix the multi-display version, still only updating the most recently created. Giving them different swapInterval values as well didn’t fix the issue, no matter which was 1 and which was 0, and no matter which order I had them flip, or having it share or not. It only ever seems to be updating the most recently created window when there’s multiple monitors. Any other ideas?