psychopy.org | Reference | Downloads | Github

Encoding data in excel

Hi, I have # -- coding: utf-8 -- at the top of my code, but when I look at my data in the output excel it still has odd signs for all of my IPA symbols. Is there a way to require the output to maintain my symbols?

So this topic can become very complicated as there are a number of ways things can go wrong.

Firstly, be aware that it should be:

#-*- coding: utf-8 -*-

But this simply makes it so you can use utf-8 IN YOUR PYTHON SCRIPT, it doesn’t affect input and output files.

The second thing I would look at is if the font excel is using can display those characters. Figure out which font is displaying them correctly when you look at your characters in other applications, and and try changing to that in excel.

You mention IPA characters, and to properly display all of them you need a special font installed on your computer. I’m attaching a pdf I made earlier this year for my phonetics class, with links and instructions for installing an IPA font (and keyboard layout), in case that’s the issue.

So before we go any further, does changing the font work?

Ah, they won’t let me upload a pdf, here’s the relevant section:

3 Install a font that contains IPA characters

Go to the SIL font download webpage to download a font (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=fontdownloads). If that link stopsworking, google “SIL IPA fonts”. Charis SIL and Doulos SIL are IPA fonts.

3.1 Windows

Follow the links to download the Windows installer, double click to install it. Restart any word processors like Word or LibreOffice so that they will recognize the font when they start up.

3.2 Mac

Follow the links to download the .zip file for Mac. Many newer Macs automatically unzip the .zip file for you, so in your Down-loads folder you should find a folder with the same name as the .zip file. If not, you can double click on the .zip file and extract it. Any file in this folder ending in .ttf is a font file. We need to drag them into the folder “~/Library/Fonts”. It’s hidden, though, so one way to get there is to select “Go > Go to folder…” from the top bar dropdown menu, type the folder name and press enter. If you had any applications open that you want to use the font, you’ll have to restart those applications for the font to appear in your menus.

3 Likes

For the sake of completeness, if simply changing the font doesn’t work, the next thing I would ask is if Excel is opening the file in the wrong encoding.

I believe psychopy will save the data in utf-8. Here is a page from Microsoft explaining how to change the encoding for a file, and it has a brief introduction to what an encoding is which is good to know.

Basically you go to File, Options, Advanced, General, and select “Confirm file format conversion on open”. Now reopen the file, and there will be some options for changing the encoding.

I very much advise that you try this with a copy of the original spreadsheet. If you open a spreadsheet in the wrong encoding, and then save it in that encoding, the file will be corrupted and you’re usually screwed.

So first try the font, then this if that doesn’t work.

2 Likes

Hi Daniel,I’m getting back to this issue since I haven’t resolved it yet. Nothing is wrong with my fonts, I have all the appropriate ones installed. I don’t see the option “Confirm file format conversion on open” in excel under the place you navigated me to. I have excel 2016 if that matters. Any other way to enforce it to open as UTF-8? I’m googling around.Stacy

I did find a work-around:

  1. Open .csv file in Notepad2. Save-as over the original file as .csv with UTF-8 encoding3. Open .csv file in Excel and voila! has the correct encoding.

:slight_smile:

That’s great! (Though you’re never overwriting the original file, right? If something goes wrong you can never go back, always mess with a copy). Glad it got figured out!