Bug: disappearing line breaks in code cells in Builder

I am finding that sometimes, Python code in some code block components gets messed up. By that I mean that all the line breaks within a code block will be removed, leading to just a single long line of python text, which obviously does not work any more. Spaces used for indentation are still present, it is just the line breaks which disappear.

My interim ‘solution’ is to manually fix it by entering line breaks again. Although this is pretty frustrating and error prone given that its crucial to get the level of indenting correct.

My setup is a mixture of Mac + PC. I am editing the .psyexp file in the builder view. Sometimes I edit on a Mac and sometimes on a PC. The files are stored in a shared dropbox folder that keeps them synced. Editing of the .psyexp only happens on one computer at a time and is only done on multiple computers because I’m either in the lab with students, or editing on laptop outside the lab.

The problem has been occurring in 1.90.1, but is also occurring now I am using 3.0.3.

I cannot work out the definitive circumstance to reproduce the problem. My hunch is that it’s some esoteric text encoding issue with Mac vs PC, but that doesn’t account for why it only affects some code blocks some of the time.

Any help or insights much appreciated.

1 Like

Thanks for letting us know which versions you are using. Could you provide a minimal working example of the bug?

I’ve found exactly the same issue recently, in the latest stand-alone release. I find that text typed into PsychoPy carries over fine between Mac and PC. The issue mostly seems to arise when I’ve been testing code somewhere else (e.g. PyCharm) or working on text in an editor like BBEdit, and then pasting it into a PsychoPy code component. It will look fine on the Mac, but just that pasted section will lose the line breaks when opened in Windows.

I wonder if it might be due to PsychoPy using Windows-style double character line breaks (CR + LF) when typing, regardless of platform, but when text is pasted in on a Mac, it comes with just a single LF character?

Yes! In retrospect, I think this is basically it. It’s always happened when I’ve pasted code into the code blocks, which has always been on a Mac. Then the line breaks disappear only on the PC, not on a Mac, as far as I can remember.

Below is a screenshot from the BBEdit text editor, where one can control the line ending character(s) on a per-document basis:

Might be worth checking whether controlling the source of the pasted text like that influences the end result. I’ll try to remember to do this next time I’m working on such a file. But to fix the problem properly would probably require making PsychoPy more aware of processing text line endings, either when it is pasted in or when the document is saved.

You can also make the EOLs visible using View > Show EOLs in the Coder menu. Also, there is a preference setting called newline convention in the Coder tab. Here, you can choose your style of newline, and that should determine how your EOL convention when reading scripts.

No luck I’m afraid. I’ve tried all combinations of pasting in text (on Mac) which was either LF or CRLF in my text editor (Atom), and Coder View > Preferences > Coder > newline convention (unix and dos). In all instances the newlines are removed when opened on the PC.

When pasting into a code block in Builder View on Mac, the pasted lines have an additional line return when pasting from CRLF format. Even when I leave in those extra blank lines we are left with zero newlines when opened on the PC.

So I think this is the right direction of enquiry, but there is no working solution as yet. As far as I can tell ALL newlines are removed when pasting into a code block on Mac (regardless of format being LF or CRLF), when opening on a PC.

PS. As far as I can tell it is not possible to ask for the EOL characters for code blocks in the Builder View.

Ok, perhaps we can look at formatting the pasted code according to the newline param. Will have a look at adding a fix on Github.

Much appreciated.

@drbenvincent, I believe the fix listed below should address your problem. Now, when a file is opened, all EOLs are converted to the EOL Coder preferences. So, if you have mixed EOL, they should now all be the same when you compile your script or open it in Coder.

1 Like

Great stuff! Thanks for the quick turnaround. I guess that should be available in the 3.0.4 release?

Yes, I would think it will be in the next release, so long as the code is ok

1 Like

Well done David, great to get this bug squashed.

1 Like

Sorry, but I don’t think this is working yet. I’m on 3.0.4 on both Mac and PC. The disappearing line break problem now occurs even just constrained to operating on a Mac alone. Yes same thing happens when you paste on a Mac and open on a PC, but now it’s somewhat worse in that any pasted text on a Mac will be inoperable when reopening that file on the same Mac.

Example 1: Newline convention = unix, source text format = LF
Paste code into a code block and you get this.


Save, quite, reopen file, and you get the disappearing newline issue. And that’s on the same computer, on a Mac.

Example 2: Newline convention = unix, source text format = CRLF
Now pasting in you get additional newlines, which seems a bit suboptimal.


Save, quit, reopen, and now both newlines disappear.

Same thing happens under newline convention = dos and both source text format types.

Sorry, but bug is still there / worse :frowning:

Ok, I have a Mac here and can recreate the error specifically when the EOL is legacy Mac, or ‘\r’. Will have another look at fixing.

1 Like

I think I have narrowed down the problem. It relates to how PsychoPy saves the Builder file. Currently, on Save, PsychoPy finds and replaces Unix style EOLS (\n) with the correct XML entry (&#10 ;), and when the Builder file is opened, vice versa, where XML entries (&#10 ;) are replaced with Unix style EOLS (\n), so, for example, the code can be formatted correctly in Builders Code Component. To fix, we will need to also check for Legacy Mac EOLs (\r) and replace with the relevant xml translation (&#13 ;).

1 Like

This should fix it:

1 Like

Hey @drbenvincent, @jon is building another release with the fix implemented. Before we go ahead, would you be able to test? This works on our Mac, but want to make sure your specific issue is addressed.

Yes, I’ve built a new version just for testing. It’s currently at
https://psychopy.org/downloads/StandalonePsychoPy3-3.0.5-MacOS.dmg
but if you find it works for you then we’ll push that to github releases as usual.

thanks for your patience,
Jon

Awesome. I’ll test it now and get back to you shortly. Thanks very much for being so quick on this bug :slight_smile: