Dear all,
I am trying to reshape English text to Arabic. Once, there was an online Arabic reshaper, but now I face “502 Bad Gateway” error trying ti open it.
I download the package from GitHub, but I don’t know how to integrate this function into Psychopy. I would appreciate any comment.
Since python-arabic-reshaper GitHub - mpcabd/python-arabic-reshaper: Reconstruct Arabic sentences to be used in applications that don't support Arabic is available by pip install, we should probably be bundling it with PsychoPy and potentially integrating some functions into our text stimuli.
But the current PsychoPy developers struggle with this, as we aren’t familiar with Arabic/Farsi/Urdu text. For example, I just ran the examples in the README at GitHub - mpcabd/python-arabic-reshaper: Reconstruct Arabic sentences to be used in applications that don't support Arabic and I can’t see what the difference is before and after the transformation (the example text doesn’t seem to match the bitmaps). We really need someone to help us with testing and implementing this.
I put out a request for this last year but the silence was deafening:
I think @jon has included the python-bidi package with PsychoPy for quite some time, so that is already available to you at least.
So let’s make this concrete: can someone (perhaps you @Omidrezaa?) please provide us with some examples:
Dear Michael,
I try to explain our problems with reshaping to Persian language.
Suppose we want to type “Welcome”. The equivalent word for “Welcome” in Persian is “خوش آمدید” and it is a two-word vocabulary. I write it in my text input or condition file as below:
And when I run the experiment, this is what I see in the screen, which is not the same word that I typed:
This statement has two problem regarded to reshaping to Persian: First, it is backward. In Ppersian we read and write from right to left (contrary to English). Thus, the first problem that we need to deal with is defining our program o write the Persian words as they are (from right to left, not backward). Second, the characters in Persian and Arabic are in isolated form, however, when they combine to each other to build a word, they change according to the surrounding characters. For example, “Welcome” in Persian, contains this characters in isolated form:
But, to make a meaningful word, they should attach to each other and some words here will change in their shapes:
This is the second issue that we need to consider when trying to solve the problem. Both od these issues are explained by Diab here:
His online reshaper worked well to solve these two problems. When I reshape strings in his website and copy them to my condition files, they were weird characters, but in the experiment, everything was clear and in the correct form.
For example, the equivalent word for “Mobile app” in Persian is:
Reshaping it in the online reshaper results in the following word:
Although this word seems weird in Persian, but if we look carefully, we will see that it has two characteristics: First, it is backward (maybe to solve the backward problem in Psychopy, this function make it backward already to neutralize the Psychopy backward function). Second, characters are reshaped according to their surrounding words. This is why we see this weird word completely correct during the experiment despite its shape in condition file.
Hope these explanations are clear.
Dear Omid,
Thanks for taking the time for such a detailed response. Could I ask you to make a Builder file where you just have a single routine with two text components one above the other, each containing that same text?
The first problem we can hopefully solve, as PsychoPy comes with python-bidi built in. Let’s say your second text component is called text_2. Insert a code component, and in its ‘Begin experiment’ tab, put this:
# library for displaying right-to-left text correctly:
import bidi
In its ‘Begin routine’ tab, put this:
# change text in the stimulus to right-to-left:
text_2.text = bidi.algorithm.get_display(text_2.text)
What happens?
OK, I’ve gone a step further and seen what the results would be if we used the Arabic reshaper library as well as the bidirectional algorithm. @Omidrezaa (or anyone else who can read this): could you please verify that the bottom line of text in the image below is the correct representation of the raw text in the top line?
Dear Michael,
regarding your first request, I have done what you asked, however, due to an error, I changed the code as follow:
from bidi.algorithm import get_display
text_2.text = get_display(text_2.text)
And this is an screenshot of the experiment screen:
It worked nicely and it solved the first problem perfectly. Now, the text is from right to left which is the right format for Persian and Arabic. However, the second problem still exists.
Now, your second answer. Yes, the last line (both) is the correct representation of the first line. Both functions (bidi and arabic_reshaper) do their job nicely.
Although I can use arabic_reshaper in IDLE, I could not import this function into psychopy (error: no module name arabic_reshaper). I would appreciate if you could explain the steps you took after using bidi.
Hi Omid, thanks for your work and independent confirmation.
The best solution would be to bundle the arabic-reshaper package with the standalone PsychoPy distributions, so that all users get access to its functions as required. I’ve submitted a request to @jon here: https://github.com/psychopy/psychopy/issues/1785
Once that package is available, it would then become very easy to actually add its functionality (and the bidirectional algorithm) to PsychoPy’s text stimulus, and ideally make it available via the graphical text component in Builder as well. We would allow them to be applied independently, as I think Hebrew users for example only need the bidirectional algorithm, whereas, as you’ve shown, languages using the Arabic alphabet need the reshaping as well.
But you don’t need to wait for that development, and can just use the functions directly, just as you’ve already done in IDLE. I ran the example above using PsychoPy within my own Anaconda Python 3 installation, not using the standalone PsychoPy application. If using one’s own Python, installing a package like this is as simple as typing pip install arabic-reshaper in a terminal. If, however, like most people, you’re using a standalone PsychoPy distribution, accessing additional packages is more complicated. Until we bundle the package, you’ll need to follow the instructions for that process here:
http://www.psychopy.org/recipes/addCustomModules.html
If you get that to work, then the code I used was this:
Begin experiment:
# library for displaying right-to-left text correctly:
from bidi import algorithm
# to reshape Arabic characters from their isolated form:
import arabic_reshaper
Begin routine:
reshaped_text = arabic_reshaper.reshape(original_text) # reshape characters
final_text = algorithm.get_display(reshaped_text) # change to right-to-left
your_text_stim.text = final_text
We’ll have anew standalone release very soon and I’ll make sure it’s in there (but it will need some info about getting the code to work). If you manage then maybe a recipe page would be good in our documentation. 
arabic_reshaper is included in 1.90.1 which is now available
Fantastic, that was a quick turn-around 
. It should be very easy now to incorporate that functionality into the TextStim, so expect a pull request on that. Similarly, it should then be straightforward to expose those settings via the Builder Text component, but I’ve never looked at extending those dialogs, so might be a bit of a learning curve.
Thank you for all of your efforts to solving the problems with Persian language. I tested the new version of PsychoPy and, as you noted, it worked well. However, there is still one unexpected problem regarding to the Persian language. I try to explain it here:
In Persian we write from right to left (contrary to English), and from up to down (similar to English). Consider the English text below:
This is the Persian equivalent of the previous English text:
Like the English text, the Persian text is only one sentence statement. However, in English text, due to the width of the text, the text are presented in 5 lines from top to down. In Persian text, similar condition appears, however, the lines are presented from down to top which is incorrect. Look the picture below for an illustration of the problem:
One minor problem is related to the alignment of the text. Since in Persian, we write and read from the right to left, align the content with the left margin give the text an awkward shape. Right or at least center alignment would make the text more elegant.
I’d just finished incorporating bidirectional and reshaping parameters directly into the TextStim, and adding examples into the Coder text stimulus demo. Was just about to submit a pull request when I saw this detailed post from you…
bidi and arabic_reshaper libraries.The question of alignment and ordering of multline text can’t be solved using pyglet text. The alignment argument in PsychoPy allows you to determine where the anchor is (i.e. where the pos refers to)
I’m afraid the only workaround for now is to break up the lines yourself and draw them in the order you want, using align=‘right’ to set the position from the right-hand edge
I think this functionality is still useful for someone who wants to present shorter sections of text.
Here is the Persian text that I used above:
در نسخه آخر سایکوپای، مشکل تبدیل متون انگلیسی به فارسی حل شد و از این پس شما قادر خواهید بود که در آزمایشات خود از متون فارسی استفاده کنید
Thank you Jon,
Yes, I think for now, breaking a long sentence and present it into different lines can solve the problem somehow.
Just a note that a new languageStyle parameter has been added to PsychoPy’s text stimulus, accessible both via Coder and in the Builder Text component. It takes three possible values at present: 'LTR' for standard Latin-style left-to-right languages, 'RTL' for right-to-left languages like Hebrew, or 'Arabic' for languages based on the Arabic alphabet, where in addition to being right-to-left, the characters need to be reshaped into cursive forms that depend on their neighbours. The 'RTL' and 'Arabic' settings both support bidirectional text (i.e. mixing portions of text, such as numbers or Latin script, in which the direction reverses to left-to-tight within the overall RTL string.)
That feature was added into the development version here:
and so should make it into the next feature release of PsychoPy. Feedback on real-world usage would be great. Issues with alignment and bottom-to-top flow have yet to be addressed.
Hello Michael and Jon,
I’m new to PsychoPy, and I have no coding exprience. I have been mainly toying around with the Builder. I have an experiment coming up with Arabic text and I would really like to use PsychoPy. I followed the discussion on reshaping English text to Arabic. My understanding that it is now accessible on both the coder and the builder text component (which is amazing! thank you!), but I’m having trouble finding it on either. My builder text component does not seem to display this parameter, I’ve uploaded screen shots of it, and on the coder end, I tried to do the following:
import bidi #i'm assuming I need to import these packages first
import arabic_reshaper #i'm assuming I need to import these packages first
# Initialize components for Routine "trial"
trialClock = core.Clock()
target = visual.TextStim(win=win, name='target',
text='تكتبين',
font='Arial',
pos=(0, 0), height=0.1, wrapWidth=None, ori=0,
color='white', colorSpace='rgb', opacity=1,
depth=0.0, languageStyle= 'Arabic')
But when I hit the run button, the participant number box appears, then the app freezes and then it displays the following error message:
################# Running: /Users/nkweider/Desktop/trial 1.py ##################
pyo version 0.8.8 (uses single precision)
2018-06-02 19:56:43.643 python[976:26454] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to /var/folders/wq/9n_jxdxj5vj6s5b8n4j3sxjr0000gn/T/org.psychopy.PsychoPy2.savedState
Exception ignored in: <bound method TextStim.__del__ of <psychopy.visual.text.TextStim object at 0x117508da0>>
Traceback (most recent call last):
File "/private/var/folders/wq/9n_jxdxj5vj6s5b8n4j3sxjr0000gn/T/AppTranslocation/0F028CE1-07DF-400E-A41C-D1B80EAFD8AF/d/PsychoPy2_PY3.app/Contents/Resources/lib/python3.6/psychopy/visual/text.py", line 177, in __del__
GL.glDeleteLists(self._listID, 1)
AttributeError: 'TextStim' object has no attribute '_listID'
Traceback (most recent call last):
File "/Users/nkweider/Desktop/trial 1.py", line 76, in <module>
depth=0.0, languageStyle= 'Arabic');
TypeError: __init__() got an unexpected keyword argument 'languageStyle'
15.5606 WARNING User requested fullscreen with size [1024 768], but screen is actually [1366, 768]. Using actual size
Do you have any recommendations or suggestions for me? I’m not really sure what I’m doing wrong. I have a macOS sierra v10.12.6 and I have downloaded PsychoPy2_PY3 v1.90.1. Thank you so much for your time and support. It is much appreciated!
Best,
Nour!
Hello Nour,
You’re not doing anything wrong. But as noted above:
i.e. the feature has not yet been included in the stand-alone release of PsychoPy, but is just currently available in the GitHub code repository for people to install themselves into their own version of PsychoPy. Typically only PsychoPy developers do this: code in GitHub is still a work in progress, and is only incorporated into the standalone releases for users every few months, once bugs have been worked out and the developers agree that new features are worthwhile and ready to be used.
So you could install your own version of Python (I’d suggest the Anaconda system for managing that), and then install the development version of PsychoPy into that. But that whole process can be quite a heavy learning curve, and really not worth doing if you don’t have much programming experience.
An easier alternative is find your current PsychoPy standalone application and edit/replace some of the relevant files there, to incorporate the new changes for this feature. On a Mac, find the PsychoPy application, right-click on it, and select "Show package contents". Then you can navigate to the Contents/Resources/lib/python3.6/psychopy or Contents/Resources/lib/python2.7/psychopy folder inside it (best to make a copy of the application before you start mucking around inside it like this).
You can find the list of edited files for this feature here:
The minimum two you would need to edit/replace are these files inside the application:
To add the feature to the TextStim:
psychopy/visual/text.py
To make the feature visible in Builder:
psychopy/experiment/components/text/__init__.py
If you aren’t in a rush, you could wait until the feature appears in the next release to users. But if you do go ahead with testing it now, any feedback would be welcome. I can’t read any Arabic/Farsi script myself (beyond the Eastern Arabic numerals I found useful to learn when travelling in Iran), so very much rely on having native speakers’ feedback on whether things are working OK.
Thank you Michael very much for your response. I attempted editing the files per your directions, but I ended up crashing the whole app -__- So I decide to just use images instead of text. My experiment is all reading isolated words and very short sentences, so I can easily create image files for each word/sentence. I really like PsychoPy and I look forward to the next release to use for future experiments, and if I can be of any help with the Arabic part, I am happy to provide any support I can.
Thank you!
Nour