Displays 30 (2009) 49–52
Contents lists available at ScienceDirect
Displays
journal homepage: www.elsevier.com/locate/displa
Review
Usability evaluation of E-books
Yen-Yu Kang a,*, Mao-Jiun J. Wang b,*, Rungtai Lin c
a
b
c
Department of Industrial Design, National Kaohsiung Normal University, 62, Shenjhong Rd., Yanchao, Kaohsiung County, 824, Taiwan, ROC
Department of Industrial Engineering and Engineering Management, National Tsing Hua University, 101, Sec. 2, Kuang Fu Road, Hsinchu 300, Taiwan, ROC
Department of Craft and Design, National Taiwan University of Arts, 59, Sec. 1, Daguan Rd., Banciao City, Taipei County 220, Taiwan, ROC
a r t i c l e
i n f o
Article history:
Received 27 June 2008
Accepted 1 December 2008
Available online 24 December 2008
Keywords:
E-book
Reading performance
Eye fatigue
a b s t r a c t
The objective of this study is to evaluate the usability of electronic books (E-books). An experiment was
designed to compare the differences between reading an E-book and a conventional book (C-book) with
objective measures. Twenty junior college students, ages sixteen to eighteen, participated in the study.
Response measures included reading performance and critical flicker fusion (CFF). The results indicate
that reading an E-book causes significantly higher eye fatigue than reading a C-book. Reading a C-book
generated a higher level of reading performance than reading an E-book. In addition, females demonstrated better reading performance than males in reading either book.
Crown Copyright Ó 2008 Published by Elsevier B.V. All rights reserved.
1. Introduction
A book is defined as a written or printed work consisting of
pages glued or sewn together on one edge and bound in covers.
It is a literary composition that is published or intended for publication as such a work [20].
Recently, the ‘‘E-book” was introduced into the market of personal digital products. Total sales in 2003 amounted to $10 million
representing an increase of more than 32% over 2002 [1]. The Ebook is a new information technology product that facilitates reading and acquisition of information. It is a written work readable on
the screen of a PC, a PDA (personal digital assistant), or a reader
specifically designed for the purpose. It provides the same meaning
as a conventional paper book (C-book) which stores and communicates knowledge through reading. On the positive side, an E-book
is superior to a C-book from diverse perspectives such as storage,
transfer, delivery, and accessibility [15]. An E-book can be described as a new platform for accessing digital information that
preserves the benefits of having electronic reading while providing
many of the advantages of paper books [19]. Further, Cox [4] indicates that an E-book can support the academic mission effectively,
saving time and adding value as a collective online reference, and
for dynamic and cost-effective collection management.
On the negative side, Doctorow [8] indicates the resolution of an
E-book is too low to effectively replace paper and that we cannot
read an E-book without power and a computer. Further, Carlson
[2] indicates that navigating through digital text is one of the big-
* Corresponding authors. Tel.: +886 3 574 2655; fax: +886 3 573 6107 (M.J.
Wang); tel.: +886 7 717 2930x7812; fax: +886 7 6051156 (Y.Y. Kang).
E-mail addresses: yenyu@nknu.edu.tw (Y.-Y. Kang), mjwang@ie.nthu.edu.tw
(M.J. Wang).
gest complaints for E-book users. They found moving from page to
page was tedious, and it was difficult to find specific chapters in
the text or to locate particular words.
Apart from the study about E-book manipulation, many studies
investigate the effect of reading performance from paper and screen.
Reading speed, accuracy, comprehension, fatigue and preference
were mainly used to evaluate reading performance [6]. Mayes
et al. [14] show that subjects take longer to read text on a screen than
on paper. As for accuracy, Egan et al. [9] report that students using
digital hypertext on a screen to find out specific information in the
text had higher accuracy than students using the paper text. For
reading comprehension, Dillon and Gabbard [7] conclude that comprehension when reading from a screen is better than reading from
paper when performing substantial searching or manipulation and
comparison of visual details among objects. However, Cushman [5]
finds that visual fatigue is significantly higher with when reading
black objects on a white screen background than reading paper. In
addition, Martin and Platt [13] also found that the medical school
students still prefer to read from paper rather than from a screen.
Most previous studies used English as the experimental text for
reading [5,7,9,13,14]. English, with an alphabetic system, is different than Chinese which has a logographic system. Each English
word includes a finite number of characters and each character
does not contain much meaning. However, a Chinese character
has its own meaning, formation and figure characteristics. When
reading Chinese from left-to-right, the region of effective vision
covers 1 character to the left of fixation to 2–3 characters to the
right of fixation [11]. But in English reading, the region extends
3–4 letters to the left of fixation to about 14–15 letters to the right
of fixation [16]. Further, the average gaze range is about 2.6 characters for reading Chinese [11] and about 7–8 letters for reading
English [16]. Rayner et al. [17] reports that among bilingual
0141-9382/$ - see front matter Crown Copyright Ó 2008 Published by Elsevier B.V. All rights reserved.
doi:10.1016/j.displa.2008.12.002
50
Y.-Y. Kang et al. / Displays 30 (2009) 49–52
Chinese/English speakers the fixation duration in the Chinese reading task is about 84 ms longer than in the English reading task. Furthermore, the effect of reading Chinese text on an E-book remains
unclear and requires further investigation.
Typically the reading habit is established in childhood. Reading
from an electronic interface is completely different than reading a
conventional book. The reading behavior and visual demand when
using an E-book require further study. Usability issues arising from
a change in reading habits need to be investigated. The objective of
this study is to evaluate the usability of reading an E-book and a Cbook by using objective measures including reading performance
and critical flicker fusion (CFF). In addition the gender effect is also
evaluated.
2. Method
2.1. Subjects
Ten male and 10 female junior college students participated in
the experiment. Their ages ranged from 16 to 18 years
(mean = 17.45, SD = 0.60). The mean age of male subjects was 17.5
(SD = 0.71). The mean age of female subjects was 17.4 (SD = 0.52).
They were required to have at least 20/25 visual acuity with corrective lenses and to be without physical or mental problems. They
were also requested not to stay up late, take medicine, alcoholic
drinks and any other substance that might possibly affect the test results. All subjects had no previous experience using an E-book.
2.2. Experimental design
This study employs a nested-factorial design. The two independent variables are book type (E-book and C-book) and gender
(male and female). Subjects were requested to read an E-book
and a C-book respectively. The E-book used in this study is illustrated in Fig. 1 and had the following characteristics: (1) viewing
screen: 17 cm 13.5 cm, (2) text: 10 cm length and 13 cm width,
(3) font size: 18-point, and (4) page format: 10 rows 18 lines
with 0.4 cm between every two rows and 0.2 cm between every
two lines. A C-book was prepared in the same format as the E-book
so as to minimize differences between the two. The aspect ratio
and paper size of the C-book was similar to the viewing screen of
the E-book. The font of both books was Ming type Chinese charac-
ters that is the most frequently used font for Chinese textual information and is also known as the standard writing type. The size of
each Chinese character in the C-book was 1.1 cm 1.1 cm which
was similar to the size of Chinese characters used in the E-book.
The main difference between reading the E-book and the C-book
was the interface manipulation. For manipulating the E-book, the
subject pressed a button to go to the next page and, of course, used
his or her fingers to turn pages in the C-book. The response measures included reading performance (speed and accuracy) and
eye fatigue measure (CFF).
2.2.1. Reading performance
In the experiment all subjects read novelettes on an E-book and in
a C-book. Each subject spent about forty to sixty minutes to read ten
novelettes page by page with an average of 2300 Chinese characters
in each novelette. There were five reading test questions after each
novelette. The test was recall-type questions. For example, ‘‘What
are all the names of the roles in the story?” or ‘‘What are all the characteristics of the roles?” Subjects were not required to search for specific information in the text. Both reading speed and reading
accuracy (proportion of correct recall) were taken as performance
measures.
2.2.2. Eye fatigue
CFF is an effective measure of visual fatigue [3,21]. It measures
the minimal number of flashes of light per second at which an
intermittent light stimulus no longer stimulates a continuous sensation. As a highly sensitive and easy-to-use measure, CFF is applied here to evaluate retina functionality. A drop in CFF value
reflects a drop in the sensory perception function, attributable to
a decrease in alertness [10].
2.3. Experiment procedure
A standard classroom desk and chair were provided for experimentation. The experiment environment was standardized. Prior
to the experiment each subject was instructed about the purpose
and procedure of the study. The sequence of novelettes was randomized for each subject. At the beginning of each reading session,
the subject’s CFF was measured as a baseline for comparison. The
subject was then asked to read ten novelettes and then take a reading recall test. Both reading speed and reading accuracy (proportion of correct recall) were measured. In addition, a visual fatigue
Fig. 1. The illustration of: (a) E-book and (b) C-book used in this study.
51
Y.-Y. Kang et al. / Displays 30 (2009) 49–52
measure was also taken. After completing one reading experiment,
another one was scheduled for one week later. Each subject was required to participate in two reading experiment sessions.
CFF (Hz)
4
3. Results and discussion
The summarized ANOVA results are shown in Table 1. The book
type effect was significant on eye fatigue. The gender effect was
significant both on reading speed and eye fatigue. Subject effect
was significant on both reading speed and eye fatigue. There was
no significant interaction effect.
3.1. Reading performance
Reading performance was measured by reading speed and reading accuracy (proportion of correct recall). Table 2 shows the average
performance of reading both books. For the C-book the average reading time was 3062.2 s and the reading accuracy was 87.8%. For the
E-book the average reading time was 3240.6 s and the reading accuracy was 88.2%. On average, reading the C-book took 178.4 s less
than reading the E-book. The reading accuracy of reading both
book-types was similar. Even though there was no significant ‘‘book
effect” on reading performance, the reading efficiency of the C-book
was somewhat higher than that of the E-book. This may be attributed to the contrast and resolution of the display in an E-book. Also,
the subjects were more used to reading C-books than E-books.
The gender effect was significant on reading speed. Male subjects spent an average time of 3239.9 s reading the C-book with a
reading accuracy of 88.4%. Female subjects spent an average time
of 2884.4 s with a reading accuracy of 87.2%. Males spent 12.3%
more time reading the C-book than females, but the reading accuracy was similar. Male subjects spent an average time of 3356.6 s
reading the E-book with a reading accuracy of 86.2% while female
subjects spent an average time of 3124.6 s with a reading accuracy
of 90.2%. Thus males spent 7.4% more time than females in reading
the E-book, but the male’s reading accuracy was 4% less. This may
be explained by the previous reports that females tend to be a
more frequent and efficient readers than males [12,18].
Male
Female
3
2
1
2.09
3.07
1.98
1.09
0
C-book
E-book
Fig. 2. The gender effect on CFF changes for both books.
had an average reduction of 1.59 Hz. After reading the E-book,
the subjects’ CFF had an average reduction of 2.52 Hz. This reveals
that reading the C-book caused less eye fatigue than reading the Ebook. This is due to the luminance contrast for the C-book was
1:8.39 and the contrast for the E-book was 1:2.90. A lower luminance contrast for the display of E-book may also contribute to a
higher eye fatigue.
Further, Fig. 2 shows the gender effect on CFF changes for both
book types. After reading the C-book, male subjects’ CFF had a
reduction of 2.09 Hz and female subjects’ CFF had a reduction of
1.09 Hz. After reading the E-book, male subjects’ CFF had a reduction of 3.07 Hz and female subjects’ CFF had a reduction of 1.98 Hz.
It seems that female subjects had significantly less eye fatigue than
male subjects due to male subjects spending a longer time reading
than female subjects.
4. Conclusion
This study evaluated the usability of reading an E-book compared with reading a C-book. The results indicate that reading an
E-book caused significantly higher eye fatigue than reading a Cbook. This is mainly due to the low contrast and resolution of the
display for an E-book. In addition, the reading efficiency for an Ebook was lower than that of a C-book. Since the reading habit for
C-books was established in childhood, people are more used to
reading C-books than E-books. Moreover, females demonstrated
better reading efficiency in both type of books than males.
3.2. Eye fatigue
Eye fatigue was measured by CFF. Book type had a significant
influence on eye fatigue. After reading the C-book, subjects’ CFF
Table 1
The ANOVA results.
Factor
Reading speed
*
**
Eye fatigue
Reading accuracy
**
CFF
**
***
*
p 5 0:05:
p 5 0:01:
p 5 0:001:
***
Table 2
The average reading performance of both books for female and male subjects.
Gender
Male
Female
Average (Sd)
Reading time (second)
Proportion of correct recall (%)
C-book
E-book
C-book
E-book
3239.9 (1149.2)
2884.4 (705.0)
3062.2 (945.6)
3356.6 (817.8)
3124.6 (571.1)
3240.6 (696.8)
88.4 (7.0)
87.2 (7.0)
87.8 (7.0)
86.2 (6.0)
90.2 (5.0)
88.2 (5.0)
(), standard deviation.
This Study was sponsored by National Science Council in Taiwan. ROC (NSC-92-2213-E-131-009-).
References
Task performance
Gender
Subject (gender)
Book type
Acknowledgement
[1] R.W. Boss, E-book; An uncertain future, Tech Notes, American Library
Association, 2004.
[2] S. Carlson, Students complain about devices for reading E-book, Study Finds,
The Chronicle of Higher Education, 2002.
[3] C.F. Chi, F.T. Lin, A comparison of seven visual fatigue assessment
techniques using three data-acquisition VDT tasks, Human Factors 40 (4)
(1998) 577–590.
[4] J. Cox, E-book: challenges and opportunities, D-Lib Magazine 10 (10) (2004)
1082–9873 (ISSN 1092-9873).
[5] W.H. Cushman, Reading from microfiche, a VDT, and the printed page:
subjective fatigue and performance, Human Factors 28 (1986) 63–73.
[6] A. Dillon, Designing usable electronic text, second ed., Taylor and Francis,
London, 2004.
[7] A. Dillon, R. Gabbard, Hypermedia as an educational technology: a review of
the quantitative research literature learner comprehension, control, and style,
Review of Educational Research 68 (3) (1998) 322–349.
[8] C. Doctorow, Ebooks: Neither E, nor books, in: Paper for the O’Reilly Emerging
Technologies Conference, San Diego, CA, 2004.
[9] D.E. Egan, J.R. Remde, T.K. Landauer, C.C. Lochbaum, L.M. Gomez, Behavioral
evaluation and analysis of a hypertext browser, in: Proceedings of CHI’89,
Association for Computing Machinery, New York, 1989, pp. 205–210.
[10] K. Hashimoto, Physiological meaning of the critical flicker frequency (CFF) and
some problems in their measurement – theory and practices of the flicker test,
Japanese Journal of Industrial Health 5 (1963) 563–578.
52
Y.-Y. Kang et al. / Displays 30 (2009) 49–52
[11] A.W. Inhoff, W. Liu, The perceptual span and oculomotor activity during the
reading of Chinese sentences, Journal of Experimental Psychology: Human
Perception and Performance 24 (1998) 20–34.
[12] W. Knulst, G. Kraaykamp, Trend in leisure time: forty years of research on
reading in the Netherlands, Poetics 26 (1) (1998) 21–42.
[13] L.A. Martin, M.W. Platt, Printing and screen reading in the medical school
curriculum: Guttenberg vs. the cathode ray tube, Behaviour and Information
Technology 20 (2001) 143–148.
[14] D.K. Mayes, V.K. Sims, J.M. Koonce, Comprehension and workload differences
for VDT and paper-based reading, International Journal of Industrial
Ergonomics 28 (6) (2001) 367–378.
[15] L. Press, From P-books to E-book, Communication of the ACM 43 (5) (2000)
17–21.
[16] K. Rayner, Eye movements in reading and information processing: 20 years of
research, Psychological Bulletin 124 (1998) 372–422.
[17] K. Rayner, X. Li, C.C. Williams, K.R. Cave, A.D. Well, Eye movements during
information processing tasks: individual differences and cultural effects,
Vision Research 47 (2007) 2714–2726.
[18] M. Rosen, Gender differences in reading performance on documents across
countries, Reading and Writing 14 (1–2) (2000) 1–38.
[19] B. Schilit, Why e-Read? Finding opportunities in the merger of paper and
computer.
Future
of
Print
Media
Journal,
1999.
<http://
www.futureprint.kent.edu/articles/schilit01.htm>.
[20] The New Oxford Dictionary of English, Oxford University Press, 1998.
[21] M.J. Wang, C.L. Huang, Evaluating the eye fatigue problem in wafer inspection,
IEEE Transactions on Semiconductor Manufacturing 17 (3) (2004) 444–447.