Home Publications Software Research Teaching Consulting Contact 
Major articles (JCR)Other articlesCitationsExternal bibliometric databases
This is the preprint version of the following article:
 
Michalski R., Grobelny J., Lindert W., (2013), The role of web layout design factors in modeling the internet user behavior, [in:] Advances in Usability Evaluation, part II, F. Rebelo AND M.M. Soares (Eds): CRC Press, Taylor & Francis Group. pp. 438-449.
 
Cited (JCR): 0, Other Cites: 0
IF:-
5yIF:-
Pt:5
 

Abstract

The presented research deals with the effects of web page design influencing the usability measured both objectively as well as subjectively. The examined factors included the background color, the number of local navigation hyperlinks, and the proportion of graphics in relation to the text on the html page. The obtained typical task execution times were first analyzed by means of the analysis of variance and later compared with the users preferences obtained by means of the Analytic Hierarchy Process technique (Saaty, 1977).

Keywords:

usability, web page design, subjective evaluation, AHP

1. Introduction

The comprehensive understanding how people search information in web sites and provision of appropriate recommendations in this regard has a huge significance not only for broadening our knowledge about the human behavior, but also is practically useful. The studies in this field give important indications for developing the guidelines in the area of human-computer interaction (HCI).

Apart from the efficiency and effectiveness of the given interactive systems, the understanding individuals likings is recently also an important issue in this field. The users preferences are connected with the user satisfaction which constitutes one of the main dimensions of the usability concepts defined both by HCI researchers and practitioners (e.g. ISO 9241, 1998; ISO 9126, 1998; Dix et al., 2004; Folmer and Bosch, 2004).

Although, there was a number of research regarding the search and click tasks (e.g. Schaik and Ling, 2001; Kalbach and Bosenick, 2003; Pearson and Schaik, 2003; Grobelny et al., 2005; Michalski et al., 2006), and preferences (e.g. Tractinsky et al., 2000; Hassenzahl, 2004; Lavie and Tractinsky, 2004; Grobelny and Michalski, 2011) separately, there were few of them that concerned both objective and subjective measures that dealt with the various web site templates commonly used in practice.

Therefore, the main goal of this study is to explore the influence of the selected factors of designing the web pages on the users operation efficiency and preference structure. For this purpose, an investigation was conducted that consisted of two stages. The first one dealt with performing the simple search and point tasks whereas the second one concerned the preference evaluation by means of the pairwise comparisons. In next passages of this work the aforementioned experiments are presented in details, analyzed and discussed.

2. Method

2.1. Participants

Thirty eight subjects participated in the efficiency examination. All of them were students of the Wroclaw University of Technology at the age between 19 and 25 years. They were not paid for taking part in the examination. Among the participants there were 22 males and 16 females. The great majority of the examined persons possessed own computer for more than three years and used Internet on a daily basis. Almost 70 percent of subjects used the Mozilla Firefox web browser, 16% Opera, 11% MS Internet Explorer, and only 5% Google Chrome. As many as 87% users used Microsoft Windows operating systems including Windows XP (54%) and Windows Vista (34%).

In the second part of the study concerned with the preferences assessment 65 students were investigated. Among them there were 33 women and 32 men. The age ranged from 19 to 28 with the average of 21 years. Most of the subjects (95%) declared using Microsoft operating systems. The Mozilla Firefox was the most popular web browser (49%), the second was Google Chrome (20%), next Opera (16%), and only 6% used MS Internet Explorer.

2.2. Apparatus

The examinations were conducted in an open-access self learning room in one of the Wroclaw University of Technology buildings. In order to prevent the attention disturbance a special box was used.

The performance tasks were carried out on one laptop computer with the external laser mouse. The resolution was set to 1024x764 pixels at its 17 inches screen. The default parameters of the screen and mouse were employed. The Windows XP Professional operating system was installed along with the Microsoft Power Point 2003 used for displaying stimuli. Free light version of the uLog (Noldus Information Technology, 2012) software was employed to record the user activity such as mouse clicks and keystrokes performed during the tests. For the preference study a different device was used. The 17 inches screen computer laptop with the 1440×900 pixels resolution. Microsoft Windows Vista Business operating system was installed together with the IrfanView software in a 4.28 version (Skiljan, 2012) used for making the comparisons.

2.3. Independent variables

Four independent variables differentiating the analyzed web site structures were manipulated: background color, the number of links in the local navigation, the proportion of the web page filled by graphics, and the way the local menu items were arranged.

The number of links in the local navigation. The number of hyperlinks were set on two levels seven (L07) and 14 (L14). According to the Miller studies (1956) a human being is capable of manage about seven simple chunks of information in the working memory, hence this number should be optimal also in specifying the quantity of links in a menu. The second level was set twice as high.

The proportion of the web page filled by graphics. This factor was also examined on two levels: 15% (G15) and 30% (G30). Those values results from the heuristic recommendation that the optimal number of illustrations is three, and they should not occupy more or less from 5% to 15% of the whole web page available space (Nielsen and Tahir, 2001). In other words, devoting more than 15% of the web page layout to the images may indicate the graphics overloading. The second level was doubled.

The local menu orientation. Two types of arranging the local menu items were used. The first, horizontal (H) one, was situated in the top section of the web page, directly under the global menu, whereas in the second arrangement they were located vertically (V) in the left hand side of the screen.

The web site templates used in this study were created generally according to the recommendations provided by Ani Phyo (2003). She considers that the typical web site contains the following modules: global, local, and administrative navigations, web page specific content (e.g. headings, subheadings, text, images, video clips, music clips, animations, captions and other objects), web page title, search tools, and page footer.

In this study there were, however, some modifications introduced. Because there are many web sites which do not separate the administrative navigation module, it was not included in our design. Additionally, there are some doubts whether those objects really help users in navigation (Nielsen, 1999). The items from the administrative navigation were located either in the local menu or the page footer. The title was also not incorporated into the research templates as this element does not matter in this investigation. Every studied web site mock-up consisted of the following six areas which were also indicated in the exemplary web sites presented in Figure 1:

  1. The global menu that contains four buttons. For the mock-up with the vertical local menu they included: registration, e-mail, forum, and chat links, while for the one with a horizontal local menu: forum, galleries, novelties, and registration. For each of the experimental condition the order of those buttons was arranged differently. This solution should prevent users from learning the buttons locations, which was not examined in this study.
  2. The local menu which included either seven or 14 elements. The smaller local menu contained: history, services, price list, picture gallery, location map, news, files to download buttons. In the case of the bigger solution the following targets were employed: science, blog, e-mail, auction, offers, games, chat, TV, help, tips, music, sport, film, and business. All of the hyperlinks were also situated at a random order for every version of the investigated web page.
  3. The content specific for the given web page - the graphics used here is different for every web page mock-up. The place devoted to the text was filled with the Lorem Ipsum words (Lorem ipsum, 2012). This Latin text allows the user to concentrate on its visual aspects instead of the meaning.
  4. The logo - there were two versions developed for this study: one for the web site versions with horizontal local menu and one for the vertical ones. The image was located in the same place for all variants.
  5. The search mechanism - it is the same size and shape in all the studied web page and is situated in approximately the same location.
  6. The web page footer - contains the same number of items for all experimental conditions and their order is also identical. The footer consists of the following hyperlinks: terms of use, contact, privacy policy, copyright information, security on the internet.

2.4. Dependent measures

The dependent variables being measured were twofold nature. The task efficiency was measured by registering the acquisition times. The time was computed from when the users clicked on the task execution order to when the target item was selected. The users preferences expressed towards the web site mock-ups were obtained by means of the application of the procedure proposed by Saaty in the AHP method (1977). The temporal parameter results were gatherer by means of the uLog program (Noldus Information Technology, 2012) that records some of the user activities in the operating system.

2.5. Experimental design and procedures

In the efficiency as well as the preferences evaluation all three independent variables were used and the combination of these factors, each one on two levels, resulted in 8 different web site designs: (2 no of local links) × (2 graphics-text proportions) × (2 local menu orientations).

All of the examined persons have never seen the research web pages before the investigation. Prior to the proper examination, the subjects were informed about the goal and the scope of the study. Then they were asked to fill in the anonymous questionnaire that included questions about the: age, gender, education, computer possessing, time spent daily for surfing on the Internet, operating system, and web browser type used. Next, the proper computer based investigation took place. The within subject experimental design was applied so every participant tested all of the experimental conditions. The two parts of the study were conducted separately. The efficiency part began by displaying on the white background the task execution order. After the mouse clicking, the given web page appeared on the screen. The user had to find and click the earlier specified target.

The proper study of determining the preference structure by means of the AHP technique started by filling out the questionnaire, which consisted of 28 comparisons, each row contained one comparison. A subject answered the question Which of the two web sites do you like more?, and indicated his/her opinion on the Likert-type scale: decidedly prettier, prettier, somewhat prettier, the same preference. Simultaneously the given pair of web sites was displayed on the computer screen. After pressing the space, next pair of web pages to compare appeared. The whole comparisons procedure lasted approximately seven minutes.

3. Results and discussion

In the sections that follow the obtained objective and subjective results analyses are provided. The gathered data are first investigated by means of the descriptive statistical parameters and then the analysis of variance was conducted for the described in previous sections independent factors. Next, the preference related data were depicted and analyzed by means of the similar statistical tools.

3.1. Objective results

3.1.1. Descriptive statistics

The average value of the task completion time for all of the experimental conditions amounted to 4.4s, with the standard deviation of 3.3s, and the mean standard error - 0.19s. The shortest time registered for the mock-ups equalled 1s whereas the longest amounted to as much as 34s. The descriptive characteristics including average acquisition times, standard deviations, mean standard errors, and extreme values for acquisition times obtained for the examined web sites are put together in Table 1. From these data, it can be easily noted that the second experimental web page mock-up was operated the fastest by the users with the mean value of 2.6s. The worst results were obtained for the eighth condition where the average acquisition time was equal 8s. The biggest dispersion of the results was observed also for the eighth layout where the standard deviation and mean standard error amounted to 5.4s and 0.88s respectively. The smallest values of these parameters were computed tor the second web page template.

3.1.2. Analysis of variance

In order to verify the significance of differences in the task completion times, a standard three way analysis of variance was employed. The calculated F statistics and respective p values for the main effects are put together in Table 2.

According to the obtained results all of the effects were statistically meaningful. Additionally, two interactions happened to be significant: the interaction between the number of local menu links with the local menu orientation, and the interaction among all of the three examined factors.

The mean task execution times along with 0.95 confidence intervals denoted by whiskers for all the statistically significant factors and interactions are presented in Figures 2-6.

These obtained outcomes for the number of links in the local menu support the theoretical expectations stemming from the Hick Hyman law (Hick, 1952; Hyman, 1953), which links the search performance times with the number of stimuli to be processed. In this study the smaller number of items in the local menu decreased the number of possible objects to be searched. This was probably the cause of the registered considerably shorter task performance times for the seven-item web mock-ups. However, it was somewhat surprising that the times performed on web pages with the 15% of graphics share were executed meaningfully faster than in the case of the layouts with the proportion of 30%. This phenomenon could be connected with the more demanding visual processing of the graphics than the text. The decidedly better completion times for the horizontal in comparison with the vertical one orientation of the local menu were already reported in some investigations (Pearson and Schaik, 2003; Michalski et al., 2006; Michalski and Grobelny, 2008) and were rather anticipated. On the other hand, the interaction between the number of links and the local menu orientation was unexpected. This result shows the superiority of the horizontal arrangement over the vertical one only for the 14 objects included in the local menu, whereas for the menu with seven-items the vertical orientation was better operated.

3.2. Preferences results and analysis

The results pertaining to the users preferences towards examined web pages are presented in this section. As it was mention earlier the preferences were examined by means of the Analytic Hierarchy approach. This technique allows for assessing the comparisons consistency level, which can be used to verify the reliability of the obtained results. The consistency ratio threshold used for this purpose was set at the level of 0.2 in this study. The application of this criterion, resulted in exclusion of 28 persons, and thus the results of 37 participants were subject to research in next sections. The relative likings are expressed as average values of the obtained AHP weights.

3.2.1. Descriptive statistics

The basic descriptive statistics of the obtained relative preferences for all examined web page templates are summarized in Table 3.

The presented results show that the markedly best perceived web page was the one with horizontal local menu consisting of seven hyperlinks and the 30% share of graphics. The worst evaluated experimental condition was the one with the vertical menu including 14 objects and the 30% of space occupied by graphical elements. It can also be observed from the data that generally better rated were web pages with lower number of local navigation links.

3.2.2. Analysis of variance

To test the significance of differences in the average weights computed for individual web sites, a standard three way analysis of variance was used. The obtained results revealed that only the number of links in the local navigation (NLN) was statistically significant merely at the level of 0.1. The rest of the analyzed factors along with all of the interactions were irrelevant. The detailed results of this analysis are demonstrated in Table 4.

The mean weights for the effect of the number of links in the local menu are illustrated in Figure 7 and show slightly higher preferences towards web pages with smaller number of menu items.

3.3. Regression analysis

The further exploration was meant to verify whether the objective results obtained in the first part of this research corresponded to the preference investigation conducted in the second phase. For this purpose the linear regression was applied and the outcome is visualized in Figure 8.

The analysis revealed no correspondence between the objective efficiency results and later subjective ratings as the R squared amounted to barely 6.3%. The lack of correlation might be possibly attributed to the fact that the subjects making the preference comparisons were not performing any tasks on the examined web pages. Therefore, their opinions expressed only the perceived attractiveness of presented layouts. In reality those preferences could be affected by the users experience in operating the given web site solution. Similar effect was observed in the study presented by Michalski (2011). It can also be noticed that in the Figure 8 there are two extreme values that probably strongly influence the regression results. If these outliers are excluded from the analysis then the linear regression happens to be significant, F(1, 4) = 7.4, p = 0.05, R2 = 65%, and takes the following form: Mean weight = 0.136 - 0.00254 x Mean time.

4. Conclusions

The present research was designed in a way that should reflect to a considerable degree the natural processes of searching and clicking the objects located in the web page. Therefore, the typical web site layouts were applied and three main factors: the number of items in the local menu, proportion of graphics to text, and the local navigation orientation were analyzed both objectively and subjectively. The objective analysis included the efficiency analysis of searching and selecting the target, while the subjective approach involved the preferences examination. Whereas the search and click investigation showed significant influence of all of the examined effects along with some interactions, the preferences were meaningfully affected (just at the level of 0.1) only with respect to number of items in the local navigation area. Such outcomes showing different impact on the studied factors depending on the objective or subjective analysis were further confirmed by the linear regression results. This may indicate that the users ratings of the viewed web sites may have little in common with the efficiency with which the web sites could be operated and emphasizes the significance of the subjective perception.

References

Dix, A., J. Finlay, G. D. Abowd, and R. Beale 2004. Human Computer Interaction. 3rd Edition, Harlow: Pearson Education.

Folmer, E. and J. Bosch 2004. Architecting for usability: a survey. The Journal of Systems and Software 70: 61-78.

Grobelny, J. and R. Michalski, 2011. Various approaches to a human preference analysis in a digital signage display design. Human Factors and Ergonomics in Manufacturing & Service Industries 21(6): 529-542.

Grobelny, J., W. Karwowski, and C. Drury 2005. Usability of Graphical icons in the design of human-computer interfaces. International Journal of Human-Computer Interaction 18: 167-182.

Hassenzahl, M. 2004. The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19, 319-349.

Hick, W. E. 1952. On the rate of gain of information. Quarterly Journal of Experimental Psychology 4: 11-36.

Hyman, R. 1953. Stimulus information as a determinant of reaction time. Journal of Experimental Psychology 45: 188-196.

ISO 9126 1998. Software product quality, Part 1: Quality model, International Standard.

ISO 9241 1998. Ergonomic requirements for office work with visual display terminals (VDTs), Part 11: Guidance on usability, International Standard.

Kalbach, J. and T. Bosenick 2003. Web page layout: A comparison between left- and right-justified site navigation menus. Journal of Digital Information 4, Article No. 153, 2003-04-28. Accessed 12 February 2011, http://journals.tdl.org/jodi/article/view/94/93.

Lavie, T. and N. Tractinsky 2004. Assessing Dimensions of Perceived Visual Aesthetics of Web Sites, International Journal of Human-Computer Studies 60(3): 269-298.

Lorem ipsum 2012, Wikipedia, Accessed February 18, 2012, http://en.wikipedia.org/wiki/Lorem_ipsum.

Michalski, R., J. Grobelny, and W. Karwowski 2006. The effects of graphical interface design characteristics on human-computer interaction task efficiency. International Journal of Industrial Ergonomics 36(11): 959-977.

Michalski, R. and J. Grobelny, 2008. The role of colour preattentive processing in human-computer interaction task efficiency: a preliminary study, International Journal of Industrial Ergonomics 38: 321-332.

Michalski, R. 2011. Examining users preferences towards vertical graphical toolbars in simple search and point tasks. Computers in Human Behavior 27(6): 2308-2321.

Miller, G. A. 1956. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review 63(2): 81.

Nielsen, J. 1999. Designing Web Usability (1st Ed.). Peachpit Press.

Nielsen, J. and M. Tahir 2001. Homepage Usability: 50 Websites Deconstructed. New Riders Publishing.

Noldus Information Technology 2012. uLog, Accessed February 18, 2012, http://www.noldus.com/human-behavior-research/products/ulog,.

Pearson, R. and P. Schaik 2003. The effect of spatial layout of and link color in web pages on performance in a visual search task and an interactive search task. International Journal of Human-Computer Studies 59: 327-353.

Phyo, A. 2003. Return on Design: Smarter Web Design That Works (1st Ed.). New Riders Press.

Saaty, T. L. 1977. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology 15: 234-281.

Skiljan, I. 2012. Irfanview, Accessed February 18, 2012, http://www.irfanview.com.

Schaik, P. and J. Ling 2001. The effects of frame layout and differential background contrast on visual search performance in Web pages, Interacting with computers 13: 513-525.

Tractinsky, N., A. S. Katz, and D. Ikar 2000. What is beautiful is usable. Interacting with Computers 13(2): 127-145.

© 2012-2017 RafalMichalski.com, Rafal Michalski All rights reserved.Last modified on: 2016-01-06HTML 4.0! standard CSS standard!