Michalski R., Koszela-Kulinska J. (2020). Eye-tracking examination of the anthropological race, gender and verbal-pictorial relative positions on ergonomics of visual information presentation. [in:] F. Rebelo & M. M. Soares (Ed.), Advances in Ergonomics in Design (pp. 23–34). LNCS Springer.
Cited (JCR): 0, Other Cites: 0
Information on how to present visual content in an ergonomic way is crucial in various fields. The current study examines how the gender, race of a person, and picture-slogan relative location influence subjects’ perceived effectiveness about a social advertisement. Relative scores of 45 participants from Wrocław University of Science and Technology were measured by binary pairwise comparisons and formally analyzed by the loglinear model for frequency data. While making comparisons, subjects’ visual activity was recorded by an eye-tracking system. Mean fixation durations were comprehensively analyzed in the context of examined factors by a standard, five-way analysis of variance: Picture race (black, yellow, white) × Picture gender (man, woman) × Picture location (left, right) × Stimulus location (left, right) × Subject gender (male, female). Subjective results regarding perceived effectiveness of the advertisement variants were compared with objective fixation durations.
eye tracking, subjective preferences, display design, brain lateralization, digital signage, visual information ergonomics, pairwise comparisons
Knowledge how to present visual information in an ergonomic way is crucial in numerous areas beginning with the design of standard control panels, their digital versions, various forms of advertising like, e.g., classic banners, design of packages and their equivalents displayed by a variety of electronic devices. It is also well known that ergonomics of presenting visual information is influenced by various factors (e.g., Brewer & Rettie, 2000; Grobelny & Michalski, 2011; Lesch et al., 2013; Ozkan & Ulutas, 2017; Santos et al, 2019; Wang et al., 2019). For instance, it has been repeatedly shown that locating verbal stimuli on the right-hand side is more effective whereas the left-hand side is better for non-verbal components (Ellis & Miller, 1981; Friedman & Polson, 1981). This type of research are still continued in multiple contexts, e.g., Cao et al. (2019) tried to answer the question if visual attention strategies of processing advertisements on web directories are more top-down or bottom-up. The ergonomics of the visual information demonstration can be evaluated both from the subjective and objective perspective. Recent developments in eye tracking methodology and technological advancements, resulting in better and better and cheaper than ever eye-trackers, allows for more in-depth analysis of human visual behavior from an objective standpoint. For a comprehensive review of the visual activity tracking research in processing advertisements, please refer to Higgins et al. (2014).
The current study constitutes another step towards improving our understanding in this regard in the context of digital presentation concerned with racial tolerance promotion and is a direct extension of our research described in Koszela-Kulińska & Michalski (2015). The main goal of the present research is to apply eye tracking methodology to examine the human visual activity depending on three factors, that is, the human race (white, yellow, black), gender (males, females), and position of the human image in relation to an advertisement's slogan (picture-text, text-picture). Specifically, we focused on identification and analysis of the level of visual attention and its distribution. Another aim was to gather once again participants’ subjective preferences and compare them with objective eye tracking measures. A review of the crucial literature on investigated in this paper factors is provided in Koszela-Kulińska & Michalski (2015).
2.1. Design of the experiment and procedure
Subjects were to visually examine a series of picture pairs and select, by a mouse click, the banner that was more convincing in their opinion. The task question “Which advertisement convinces you more?” (Polish: “Która reklama bardziej Cię przekonuje?”) appeared at the top of each slide, see a sample comparison in Fig. 1. All banners presented a human image together with the campaign text: “Racism? No, thanks.” (Polish: “Rasizm? Nie, dziękuję!”).
Fig. 1. A sample comparison. Faces were not blurred during comparisons.
Three factors were explored. The effect of the human race of a person presented in the advertisement was specified on three levels, that is, white, black, yellow. The gender of the visible person involved two levels (man, woman). Two levels were also employed for the picture-slogan relative location, namely: human picture either on the left or on the right hand side of the slogan text.
We used the same images as those employed in our previous research (Koszela-Kulińska & Michalski, 2015). Five students visible in these pictures gave an informed consent for using the photographs for research purposes. The black female picture was bought on http://www.shutterstock.com. All images present persons with a neutral facial expression, casually dressed, and standing upright with their arms along a trunk. All the images are presented in Fig. 2.
Fig. 2. Pictures of persons used in the study. Faces were not blurred during comparisons.
Since the full factorial, within subject design was employed, all subjects assessed all twelve experimental conditions: 3 (Races) × 2 (picture gender) × 2 (image-text relative location). Each participant evaluated all possible pairs of investigated stimuli, that is, (122 – 12) / 2) = 66. The data gathered in this way were used to formally verify whether participants’ relative likings were significantly influenced by the investigated factors.
While performing randomly presented pairwise comparisons, objective visual behavior data were collected for all participants. For this purpose, a modern SMI RED500 (SMI) stationary infrared eye-tracker system was applied. The system records eye ball movements at 500 Hz sample rate with 0.4° accuracy. SMI BeGaze 3.7 was used for exporting raw fixation data for analyses in an external package. All the statistical analyses were performed in Statistica, version 13 (TIBCO Software Inc.).
The main part of the experiment was preceded by a short electronic questionnaire regarding participants’ characteristics followed by an appropriate 5-point calibration and validation procedure. The examination was held in a laboratory environment that has an isolated room equipped with a desk, typical office chair, keyboard, optical computer mouse, and 21 inches monitor. Subjects’ behavior was monitored through a one way mirror and registered by video cameras. Communication with participants took place via a set of microphones and speakers. SMI Experiment Center 3.7 software was employed to prepare and control the experiment. The software randomized the comparisons’ display order. The stimuli locations on the screen were also randomized and counter balanced.
Overall, 45 white participants, at the age between 18 and 25 years, took part in the experiment (mean = 20.3, SD = 1.18). The sample was balanced with respect to gender. There were 22 males and 23 females students of Wrocław University of Science and Technology, Poland. Mean eye tracking ratio equaled 96.2% (SD = 4.28). There were no difference between mean eye tracking ratio for males and females (F(1, 43) = 0.293, p = 0.59). Three persons (2 men and 1 woman) were excluded from further analyses as their eye tracking ratios were lower than 90%. Another three subjects were not included in analyses since too big mean deviations (> 1.5°) were discovered while examining validation procedure results. Finally, eye tracking data from 39 participants, including 18 males and 21 females, were examined in the section regarding visual activity.
3.1. Preferences – pairwise comparisons
3.1.1. Basic statistics
Subjects preferences were collected by binary pairwise comparisons. The registered selections for all three examined effects along with the stimulus location on the screen are put together in Table 1 and graphically presented in Fig. 3. The data suggest that participants were generally more convinced by conditions including a white person, less persuaded by a yellow individual, and the least – by picture with the black one. It can also be observed that advertisements with men were chosen more often if the stimulus appeared on the right hand side of the screen whereas graphical panels with women this phenomenon was reversed. The option with white man presented on the right side of the advertisement and displayed on the right was selected the most frequently. The variant with black women on the left, presented on the right hand side of the screen was the least convincing.
Table 1. Frequency statistics for all examined factors.
No. Picture gender Picture
race Picture location in relation to text Stimulus on the left Stimulus on the right
1. Man (M) White (W) Left (PL) 169 172
2. Man (M) White (W) Right (PR) 155 205
3. Man (M) Black (B) Left (PL) 79 97
4. Man (M) Black (B) Right (PR) 79 96
5. Man (M) Yellow (Y) Left (PL) 114 138
6. Man (M) Yellow (Y) Right (PR) 104 150
7. Woman (W) White (W) Left (PL) 167 162
8. Woman (W) White (W) Right (PR) 170 153
9. Woman (W) Black (B) Left (PL) 81 58
10. Woman (W) Black (B) Right (PR) 71 63
11. Woman (W) Yellow (Y) Left (PL) 137 108
12. Woman (W) Yellow (Y) Right (PR) 137 105
Fig. 3. Frequencies of selections for all examined factors
3.1.2. Loglinear analysis
A loglinear analysis was applied to verify whether the observations made in the previous section were statistically significant. The analysis involved five effects. Three main experimental factors were examined: picture race (PR: white, black, yellow), picture gender (PG: man, women), picture-text location (PL: left, right), along with the location of the presented advertisement on the screen (SL: left, right) and subject gender (SG: male, female). All possible interactions between these factors were also included in the initial model. In the resulting contingency table, minimum and maximum cell frequencies amounted to 22 and 111, respectively.
After applying the optimization procedure the recommended model involved one main effect: Picture race and two interactions: Picture gender × Subject gender, and Picture gender × Stimulus location. The maximum likelihood Chi-square value amounted to 28.6, with p = 0.91, and 40 degrees of freedom. The convergence was reached after two iterations with the criterion set at 0.01.
The selection frequencies for the significant Picture race factor are given in Table 2 and graphically demonstrated in Fig. 4. The data confirm the conjecture made in the previous section. They show the highest preferences when white individuals are presented in the advertisement. Black persons were the least persuasive for subjects.
Table 2. Number of selections for three examined races
Picture race No of selections
Fig. 4. The influence of Race on the number of advertisement selections.
The number of advertisement selections depending on the Picture gender × Subject gender interaction are put together in Table 3 and illustrated in Fig. 5. We can observe that male subjects differently perceived tested advertisements than females. For females, the gender of the person presented in the stimulus was important. They, generally, preferred variants with men over those with women. Male subjects seem to pay no attention to the person gender.
Table 3. Number of selections for the Picture gender × Subject gender interaction
Picture gender Total
gender Male 730 722 1452
Female 828 690 1518
Total 1558 1412 2970
Fig. 5. The influence of Picture gender × Subject gender on subjects’ preferences.
The influence of the Picture gender × Stimulus location interaction on subjects preferences is demonstrated in Fig. 6. Corresponding frequencies are provided in Table 4. The results suggest that when a stimulus appeared on the left part of the screen, subjects tend to select advertisement with women. If the stimulus was demonstrated on the right, the situation was reversed: variants with men were more preferred.
Table 4. Number of selections for the Stimulus
location × Picture gender interaction.
Picture gender Total
Stimulus location Left 730 722 1452
Right 828 690 1518
Total 1558 1412 2970
Fig. 6. The influence of Stimulus location × Picture gender on subjects’ preferences.
3.2. Eye-tracking results
Among numerous visual activity parameters, the fixation durations are among the most interesting and most often analyzed. Longer fixations usually mean that subjects’ visual attention was drawn to a specific part of the stimulus. Heat maps prepared for all the 66 pairwise comparison slides displayed in the current investigation exhibit very similar pattern. A sample of such a heat map is presented in Fig. 7. These data clearly show that participants generally focused on presented faces while considerably shorter average fixation times were detected on slogans. The top task order text drew attention only sporadically. Long fixations visible at the center of the comparison resulted from focusing on the cross that was displayed before each advertisement appeared.
Fig. 7. Sample heat map with average fixation durations for one of the examined comparisons.
A five-way analysis of variance Picture race (PR) × Picture gender (PG) × Picture location (PL) × Stimulus location (SL) × Subject gender (SG) was employed to verify if the mean fixation durations were significantly influenced by these effects. The obtained results regarding significant factors and two-way interactions are put together in Table 5. All other interactions were statistically insignificant.
Table 5. Five-way Anova results for significant factors and interactions.
Effect Degrees of
freedom Mean sum
of squares F p η2
Picture race (PR) 2 46363 2.36 0.0942** 0.00024
SL × SG 1 115654 5.89 0.0152* 0.00030
PG × SG 1 59676 3.04 0.0812** 0.00015
SL × PL 1 56686 2.89 0.089** 0.00015
Error 19707 19626
* p < 0.05
** p < 0.1
The analysis revealed statistically significant influence of Picture race, and three interactions: Stimulus location × Stimulus gender, Picture gender × Stimulus gender, and Stimulus location × Picture location on mean fixation durations recorded for a given experimental condition. The Picture race effect, illustrated in Fig. 8 (whiskers in all figures denote 95% confidence intervals) shows that the least visual attention was devoted to white models, whereas yellow models were explored the most intensely. The LSD Fisher post-hoc tests (Table 6) revealed that the difference only between these two levels were statistically meaningful.
Table 6. Results of LSD Fisher post-hoc tests for
Picture race mean fixation durations
Picture race White Black Yellow
Black 0.26 ×
Yellow 0.034 0.31 ×
Fig. 8. Influence of Picture race on mean fixation durations
Table 7 contains the LSD Fisher’s pairwise comparisons for the Stimulus location × Subject gender interaction (Fig. 9). Only in two cases the differences occurred to be significant. For right stimulus location, females’ mean fixations durations were longer than the males’ ones. Females paid more visual attention when the stimulus appeared on the right than on the left hand side of the screen.
Table 7. Results of LSD Fisher post-hoc tests for Stimulus location × Subject gender mean fixation durations
Stimulus location Left Right
gender Male Female Male Female
Left Male ×
Female 0.27 ×
Right Male 0.41 0.81 ×
Female 0.17 0.015 0.034 ×
Fig. 9. Influence of Stimulus location × Subject gender on mean fixation durations
The Picture gender × Subject gender interaction demonstrated in Fig. 10 seems to be quite week. The analysis of post-hoc comparisons given in Table 8 suggests that there is only one meaningful (α = 0.1) difference: the average fixation durations for female subjects were longer for advertisements with women than with men.
Table 8. Results of LSD Fisher post-hoc tests for
Picture gender × Subject gender mean fixation durations
gender Men Woman
gender Male Female Male Female
Man Male ×
Female 0.53 ×
Woman Male 0.65 0.86 ×
Female 0.31 0.10 0.14 ×
Fig. 10. Influence of Picture gender × Subject gender on mean fixation durations.
The Stimulus location × Picture location interaction is graphically illustrated in Fig. 11. Post-hoc pairwise comparisons that are put together in Table 9, indicate two significant differences. When stimulus is on the right, the advertisements with pictures located on the left attract visual attention for longer than those with pictures placed on the right side of the slogan. Variants with left positioned pictures have also longer fixations when the stimulus was displayed on the right than on the left side of the monitor.
Table 9. Results of LSD Fisher post-hoc tests for Stimulus location × Picture location mean fixation durations
location Left Right
Picture location Left Right Left Right
Left Left ×
Right 0.85 ×
Right Left 0.048 0.068 ×
Right 0.79 0.65 0.030 ×
Fig. 11. Influence of Stimulus location × Picture location on mean fixation durations.
4. Discussion and conclusion
The present research was focused on exploring visual attention recorded while performing binary pairwise comparisons of social advertisements. Digital versions of banners were differentiated by three factors, i.e., race and gender of a person picture along with the text-picture relative position. The stimulus location on the screen and subject gender were counterbalanced in this study, and they were also included in analyses.
The loglinear analysis of selection frequencies showed that white models were the most convincing, while the smallest impact was noted for advertisements with black persons. The race effect occurred to be statistically significant for the fixation duration analysis. It appears that the most convincing advertisements with white persons were associated with the shortest mean fixation durations. Such a result may be related with the subjects’ race. All participants were white and, probably, other races were deemed more interesting for them, all the more that in Poland black and yellow people are very scarce. However, it is not clear why the longest average fixations were observed for adverts with yellow models. This will require supplementary research. Subsequent studies could also include other than white subjects.
Female subjects were more persuaded when a man appeared on the stimulus. Again, similar to the race effect, correspondence between preferences and eye tracking data was noticed. The interaction revealed that females processed longer stimuli with women, but these advertisement were less convincing for them than variants with men. For stimuli located on the right, versions with a man were more convincing, and when the stimulus appeared in the left part, subjects were inclined to choose conditions with a woman model. These subjective results have no significant correspondence with the attention distribution measured in the present research only by fixation durations. However, it is not excluded that there are some relations with other visual activity measures (Goldberg & Kotval, 1999; Grobelny et al., 2006), which should be further examined.
The analysis of eye-tracking data provided also additional insight of subjects’ visual behavior that has not got its equivalent in presented subjective scores. The strongest influence on mean fixation durations (the biggest η2 value) had the interaction of Stimulus location and Subject gender. For right stimulus location, females’ mean fixations durations were longer than the males’ ones. Furthermore, females paid more visual attention when the stimulus appeared on the right than on the left hand side of the screen.
The obtained subjective outcomes seem to be in a contradiction with our previous findings (Koszela-Kulińska & Michalski, 2015) where advertisements with a black person were decidedly the most preferred, and participants liked more female models than male ones. The discrepancy probably comes primarily from the different question used. In the earlier study, we asked “Which advertisements is prettier?”, whereas in the present investigation the question was “Which advertisement convinces you more?”. Apparently, these two tasks are not identical and subjects may consider prettier advertisements as less persuading. This finding can be pursued further in future research, however, both researchers and practitioners should be very precise and cautious while specifying experimental tasks.
Another source of inconsistency between these two studies is concerned with the sample. Previously, we examined students from the Academy of Art and Design whereas here participated students from the technical university. Considerably different profiles of these two groups could have influenced outcomes. Moreover, in contrast to the current study, the previous sample was not counterbalanced with regard to the subject gender. Therefore, the interaction Picture gender × Subject gender observed in the present research could not be detected in the previous investigation, due to the highly unbalanced Anova (23 females versus 11 males).
The outcomes of the visual activity distribution are, generally, not controversial. The reported focus on the person face has been repeatedly confirmed in numerous studies (e.g., Yarbus, 1965). Similarly, the visual concentration on the panel center during general comparisons of various stimuli was also reported (e.g., Djamasbi et al., 2011).
Presented and discussed in this research results, should be treated with appropriate reserve especially due to the moderate in size, and highly homogenous sample. However, demonstrated findings may be helpful in designing more effective social advertisements and they certainly constitute a next step in better understanding visual behavior in the marketing context. Further research directions can also involve more sophisticated approaches for modelling visual data such as Markov models (c.f., Grobelny & Michalski, 2017a, 2017b).
1. Brewer C., Rettie R. (2000) The verbal and visual components of package design. The Journal of Product and Brand Management, 1, 56-70.
2. Cao, Y., Qu, Q., Duffy, V. G., & Ding, Y. (2019). Attention for Web Directory Advertisements: A Top-Down or Bottom-Up Process? International Journal of Human–Computer Interaction, 35(1), 89–98. https://doi.org/10.1080/10447318.2018.1432162
3. Djamasbi S., Siegel M., Tullis T. (2011). Visual Hierarchy and Viewing Behavior: An Eye Tracking Study. In: J. A. Jacko (Ed.), Human-Computer Interaction. Design and Development Approaches (pp. 331–340). Springer.
4. Ellis A.W., Miller D (1981) Left and wrong in adverts: Neuropsychological correlates of aesthetic preference. British Journal of Psychology, 2, 225-229.
5. Friedman A, Polson M.C. (1981) Hemispheres as independent resource systems: limited-capacity processing and cerebral specialization. Journal of Experimental Psychology: Human Perception and Performance, 5, 1031-1058.
6. Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24, 631–645.
7. Grobelny, J., Michalski, R. (2011). Various approaches to a human preference analysis in a digital signage display design, Human Factors and Ergonomics in Manufacturing & Service Industries, 21(6), 529-542. https://doi.org/10.1002/hfm.20295
8. Grobelny J., Jach K., Kuliński M., Michalski R. (2006). Śledzenie wzroku w badaniach jakości użytkowej oprogramowania. Historia i mierniki. In: Materiały konferencyjne: Interfejs użytkownika – Kansei w praktyce, (pp. 1–9). Warszawa: Wydawnictwo PJWSTK. https://repin.pjwstk.edu.pl/xmlui/handle/186319/166.
9. Grobelny J., Michalski R. (2017a). Applying Hidden Markov Models to Visual Activity Analysis for Simple Digital Control Panel Operations. In: J. Świątek, Z. Wilimowska, L. Borzemski, A. Grzech (Ed.), Advances in Intelligent Systems and Computing (pp. 3–14). Springer. https://doi.org/10.1007/978-3-319-46589-0_1
10. Grobelny J., Michalski R. (2017b). Zastosowanie modeli Markowa z ukrytymi stanami do analizy aktywności wzrokowej w procesie oceny wirtualnych opakowań techniką porównywania parami. Zeszyty Naukowe Politechniki Poznańskiej. Organizacja i Zarządzanie, 73, 111–125. https://doi.org/10.21008/j.0239-9415.2017.073.08
11. Higgins, E., Leinenger, M., & Rayner, K. (2014). Eye movements when viewing advertisements. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.00210
12. Koszela-Kulińska, J., & Michalski, R. (2015). The Effects of the Anthropological Race, Gender and Location of Verbal-Pictorial Stimuli on the Usability of Visual Information Conveyance. In M. Kurosu (Ed.), Human-Computer Interaction: Users and Contexts (441–451). Springer. https://doi.org/10.1007/978-3-319-21006-3_42
13. Lesch, M. F., Powell, W. R., Horrey, W. J., & Wogalter, M. S. (2013). The use of contextual cues to improve warning symbol comprehension: making the connection for older adults. Ergonomics, 56(8), 1264–1279. https://doi.org/10.1080/00140139.2013.802019
14. Ozkan F., Ulutas, H.B. (2017). Using eye-tracking data to evaluate medicine information leaflets on-screen. Journal of Mathematics and Statistical Science. 3(12): 364-376.
15. Santos, C., Neto, M. J. P., & Neves, M. (2019). The Influence of Infographics in Accessing Information: Multidimensionality in Visual Representation and Configuration of Different Media. In F. Rebelo & M. M. Soares (Eds.), Advances in Ergonomics in Design (pp. 497–508). Springer International Publishing.
16. Wang Y., Liu Q., Lou W., Xiong D., Bai Y., Du J., X. Guo (2019). Ergonomics Evaluation of Large Screen Display in Cockpit Based on Eye-Tracking Technology. In S. Long & B. S. Dhillon (Eds.), Man-Machine-Environment System Engineering (347–356). Springer.
17. Yarbus A.L. (1965). Eye Movements and Vision, Plenum Press, New York.