Original question asked by Giulio: “It seems we look at the left side of people’s faces only. But do people who read right-to-left (e.g. Jews, Arabs) see the right side instead?”
The effect described in this question is what’s known as the ‘left hemiface bias’, and is essentially the observation that the majority of healthy people seem to show a preference for information present in the left half of a face (or ‘hemiface’) compared to the right half (from the observer’s perspective). To demonstrate this, and to show you how this is often investigated experimentally, let’s look at some faces!
Below you’ll see two faces, one on top of the other. These faces are known as ‘chimeric’ faces, which simply means the left side is different to the right side. Participants in these experiments are shown faces like this (though they are often much better quality these days!) and asked to decide which face displays the target characteristic the most. In the below case, for example, the experimenter might ask ‘which face is happier?’ Perhaps what’s been written so far has given away the result, but have a look at the faces below and decide which you think is happier!
Figure 1. Two gruesome-looking pictures of the author. One half of each face is neutral in expression, while the other is happy. Participants more often indicate that the face displaying the smile on the left side (top image) is happier overall, though both are essentially identical. The above have been styled to look like the classic chimeric faces test stimuli by Levy et al. (1983) .
What you might have already spotted is that the two faces are identical, and are simply mirror images. Researchers find however, that over many trials, participants show a reliable bias towards saying that the face showing the target characteristic (‘happiness’ in this case) in the left side of the face, is happier overall. Of course, this is a very subtle effect and not one we’re usually aware of – we don’t only look at the left of people’s faces (at the time of writing I have a slight bruise under my eye on the right side; you wouldn’t miss that!). However, the observation is very robust [2, 3], and is often interpreted as being a result of an asymmetry in the function of our brain; because the left side of visual space (as thus often the left of a person’s face) is processed by the right hemisphere, researchers argue the effect represents a right hemisphere ‘speciality’ in face processing.
However, the point raised here about reading direction is a very clever one, and an excellent example of ‘Occam’s Razor’ style thinking which is vital in psychology. After all, why should we assume the majority of people have a strange asymmetry in their brain if a simpler explanation is available? Maybe the effect is just a result of scanning habits picked up from reading in a certain way. Experimenters have investigated this in exactly the way the author of this question described, with very clear hypotheses.
- Take one sample of people for whom it is natural to read left-to-right.
- Take another sample who normally read right-to-left.
- Compare the biases that they display.
- If the left hemiface bias is completely independent of reading direction, then both groups should show no difference in their biases.
- If reading direction is the thing that determines your bias, however, the two samples of participants should be biased towards completely opposite sides of the face.
As is often the case in psychology, the answer lies somewhere in the middle! It’s generally found that the leftward bias is reduced (but not eliminated) for participants who read right-to-left [4,5]. According to a fairly recent study on the problem , where participants matched faces to stimuli presented briefly in their left and right visual field, both left-to-right and right-to-left groups showed a left side bias, but for the right-to-left readers this bias was much weaker. In other words, our brains do seem to have a right hemisphere speciality for processing information from faces (which fits nicely with the results of neuroimaging research ), but the strength of the effect can be modulated by other factors, like reading direction. Another approach entirely has been to look at animal’s preferences (and eliminate any reading bias entirely). Curiously, the left bias effect also seems to be observed in chimps! 
It might also be interesting for you to know as well that this leftward bias also occurs in our visual attention, and can be measured by a very similar task as shown below. Try and guess which of these two bars people most often report is darker (if you can guess correctly, then I’ve done a decent job of explaining everything so far!):
Figure 2. An example of stimuli from the ‘Greyscales’ task of attentional biases. Over many trials, most participants show a bias whereby the bar which has the dark shading to the left (bottom bar in this example) is darker overall. This figure is edited from a screenshot taken within the free software from Flinder’s University.
Yes, more often people say it’s the bar where the dark side is on the left (the bottom bar in the picture above). If you flip the question, and ask which bar is lighter, then people more often pick the top bar.
What I have studied in my research, similar to the problem of reading direction, is to what extent the left bias for faces is affected or modulated by this leftward bias in our visual attention. You can check out my latest publication here.
Check out Perception Lab at University of St. Andrews and Face Lab at University of Glasgow for more information on cool face research like this. For more on the structural and functional differences between the brain hemispheres, check out this book.
Levy, J., Heller, W., Banich, M. T., & Burton, L. A. (1983). Asymmetry of perception in free viewing of chimeric faces. Brain and Cognition, 2, 404-419. (example task stimuli)
Bourne, V. J. (2010). How are emotions lateralised in the brain? Contrasting existing hypotheses using the chimeric faces test. Cognition and Emotion, 24(5), 903-911.
Burt, D. M., & Perrett, D. I. (1997). Perceptual asymmetries in judgements of facial attractiveness, age, gender, speech and expression. Neuropsychologia, 35, 685-693.
Gilbert, C., & Bakan, P. (1973). Visual asymmetry in perception of faces. Neuropsychologia, 11, 355-362.
Vaid, J., & Singh, M. (1989). Asymmetries in the perception of facial affect: Is there an influence of reading habits? Neuropsychologia, 27, 1277-1287.
Megreya, A. M., & Harvard, C. (2011). Left face matching bias: Right hemisphere dominance or scanning habits? Laterality, 16, 75-92.
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialised for face perception. Journal of Neuroscience, 17, 4302-4311.
Dahl, C. D., Rasch, M. J., Tomonaga, M., & Adachi, I. (2013). Laterality effect for faces in chimpanzees (Pan troglodytes). Journal of Neuroscience, 33, 13344-13349.