PSU Research Review
Teaching evaluat ion and st udent response rat e
Tashfeen Ahmad,
Article information:
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
To cite this document:
Tashfeen Ahmad, (2018) "Teaching evaluation and student response rate", PSU Research Review,
https://doi.org/10.1108/PRR-03-2018-0008
Permanent link t o t his document :
https://doi.org/10.1108/PRR-03-2018-0008
Downloaded on: 30 Oct ober 2018, At : 08: 45 (PT)
Ref erences: t his document cont ains ref erences t o 39 ot her document s.
Access t o t his document was grant ed t hrough an Emerald subscript ion provided by All users group
For Authors
If you would like t o writ e f or t his, or any ot her Emerald publicat ion, t hen please use our Emerald
f or Aut hors service inf ormat ion about how t o choose which publicat ion t o writ e f or and submission
guidelines are available f or all. Please visit www. emeraldinsight . com/ aut hors f or more inf ormat ion.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and pract ice t o t he benef it of societ y. The company
manages a port f olio of more t han 290 j ournals and over 2, 350 books and book series volumes, as
well as providing an ext ensive range of online product s and addit ional cust omer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.
*Relat ed cont ent and download inf ormat ion correct at t ime of download.
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/2399-1747.htm
Teaching evaluation and
student response rate
Tashfeen Ahmad
Teaching
evaluation and
student
response
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
University Project Management Office,
University of the West Indies, Kingston, Jamaica
Abstract
Purpose – The purpose of this paper is to share the author’s viewpoint on how to increase student response
Received 4 March 2018
Revised 21 June 2018
6 July 2018
Accepted 10 July 2018
rate in course evaluation surveys.
Design/methodology/approach – The approach is to highlight measures which increased student
response rate in online surveys of the author’s teaching evaluation at The University of the West Indies, Jamaica.
Findings – This viewpoint suggests that student response rate to course evaluation can be improved by the
lecturer’s effective communication. The examples of effective communication are given in this paper.
Originality/value – This work will encourage the lecturers to initiate more student engagement to
improve response rate of their teaching evaluation.
Keywords Online survey, Course evaluation, Paper survey, Response rate, Teaching evaluation
Paper type Viewpoint
Importance of student evaluation of teaching
Student evaluation of teaching is important for a number of reasons. These evaluations
ensure quality in university teaching, provide an independent method of gauging teacher’s
effectiveness, guide in making decisions for major curriculum changes and professional
development for faculty and help in establishing a framework to better quantify and reward
good teaching outcomes.
Shift in student evaluation of teaching from paper-based to online surveys
Paper-based assessment has been the most common form of student evaluation of
teaching worldwide. However, over the past decade, there has been a shift away from
paper-based to online assessment. As internet is becoming more available and affordable,
traditional paper-based data collection methods, seem expensive, time consuming and
less efficient.
Positives of this shift to online evaluation
One of the most important positives is efficiency gains, in terms of turnaround time from
students and significant cost savings. In addition, online evaluations allow students the
time, ease and ability to refine, expand and reflect on responses without the constraint of an
“in class” time bound environment to complete paper-based surveys. This increases student
© Tashfeen Ahmad. Published in PSU Research Review: An International Journal. Published by Emerald
Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence.
Anyone may reproduce, distribute, translate and create derivative works of this article (for both
commercial and non-commercial purposes), subject to full attribution to the original publication and
authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
PSU Research Review
Emerald Publishing Limited
2399-1747
DOI 10.1108/PRR-03-2018-0008
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
PRR
response to open-ended questions which provide qualitative data which is instrumental in
improving teaching practices.
A review of the major literature works over the period 2000-2013 (as depicted in Table I)
summarizes the main advantages of online method of evaluation.
Challenges of this shift to online evaluation
One of the biggest challenges is the low response and return rate of students to online
evaluations when compared to paper-based evaluation responses (Benton et al., 2010;
Goodman et al., 2015; Guder and Malliaris, 2010; Nowell et al., 2010).
A sample of the findings of response rates drawn from different research studies at
various higher education institutions (as seen in Table II) over the years 1999-2013 indicates
Authors (Year)
Main advantages
Hmieleski and
Champagne (2000)
Kasiar et al. (2002)
More written feedback
Refine, reflect, expand on responses
More written feedback
Refine, reflect, expand on responses
Johnson (2002)
Richer and higher data collection
Hardy (2003)
More written feedback
Refine, reflect, expand on responses
Ballantyne (2003)
Richer and higher data collection
Ballantyne (2004)
Richer and higher data collection
Anderson et al. (2006) Provide more feedback
Richer and higher data collection
Donovan et al. (2006) Provide more comments about
Lecturer
Laubsch (2006)
More written feedback
Refine, reflect, expand on responses
Donovan et al. (2006) Richer and higher data collection
Emery et al. (2008)
Table I.
Main advantages of
using online course
evaluation surveys
Miller (2010)
Samuels (2013)
Authors (Year)
Layne et al. (1999)
Sax et al. (2003)
Dommeyer et al. (2004)
Table II.
Anderson et al. (2005)
Comparison of
Avery et al. (2006)
response rates (online Laubsch (2006)
versus paper-based
Nair et al. (2008)
evaluation)
Perrett (2013)
Efficiency, cost savings, richer
responses
Time and cost savings, richer
responses
Richer responses, efficiency, quicker
and cost savings
Research focus areas
Student course evaluations
Comparison of traditional and Web-based
evaluation processes
Online student ratings
Online student ratings
Online evaluations of teaching
Online student survey and comments
Student course evaluations
Student feedback on online vs traditional
course evaluations
Comparison of online and in person
evaluations
Constructive student feedback on online and
traditional evaluations
Open source online evaluation experiences
Online evaluations
Academic departments use of online course
evaluations
Institution
Response rates
Southeastern University
Several US institutions
California State University
University of Kentucky
Cornell University
Fairleigh Dickinson University
Monash University
Large university in South US
47% – online vs 60% – paper
17% – online vs 24% – paper
43% – online vs 75% – paper
83% – online vs 80% – paper
47% – online vs 69% – paper
61% – online vs 82% – paper
31% – online vs 56% – paper
71% – online vs 68% – paper
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
general lower return rates for online evaluations compared to paper-based evaluations in all
except two cases (ranging as low as 17 per cent to a high of 83 per cent).
This low number of response rates, in online surveys, makes the data invalid. To
mitigate this challenge, Nulty’s (2008) research provides a set of guidelines for required
response rates to be considered valid and useful measure of accuracy for online evaluation.
Since the larger is the class size, the lower response rate is required, Nulty recommends an
ideal required response rate of 58 (size <20) and 35 per cent (>50) for accuracy of online
survey results and to achieve validity.
Major reasons for the differences in response rates
The reasons for the differences in response rates range from gender and age factors (Hatfield
and Coyle, 2013); privacy and anonymity (Khorsandi et al., 2012; Nevo et al., 2010); social
pressure; distraction and location issues (Mau and Opengart, 2012); lack of engagement;
incentives; communication; perceived inaction with feedback or general “survey fatigue”
(Bennett and Nair, 2010); and demographic and economic variables peculiar to the
institution of country (Morrison, 2011).
Solving the issue of low response rate
Bennett and Nair (2010) in their study at an Australian University were able to register an
overall 83 per cent online response rate, but this was in response to the deliberate strategies
and measures implemented to increase student involvement. Using effective engagement,
communication and teacher–student participation techniques led to greater and more
sustained response rates.
Measures to increase student online response rates
A vast amount of literature has been written about the problems and the strategies which
can be used to encourage and increase the response rates of student online evaluation (Crews
and Curtis, 2011; Morrison, 2011; Stowell et al., 2012).
The most comprehensive work done by Berk (2012) outlines a review of the problems
and articulates an in-depth set of techniques and best practices which can be applied to
increase online response rates. It should be noted however that he does not advocate a “one
size fit all” solution but emphasizes that success in raising response rates will most likely be
met by a combination of strategies and incentives over the long term.
In my opinion, the most important and fundamental ingredients for raising online
response rates depends to large extent on the commitment, engagement and buy in of both
students and teaching administrators to the process. For example, studies indicate that the
biggest determinant for student participation in online evaluation is the level of engagement
they obtain from teachers (Gaillard et al., 2011).
Those institutions which take the time to communicate and explain the process, how
their responses will be used or incorporated to improve course delivery and outcomes
experience increase in response rates (Wode and Keiser, 2011). On the other hand, students
who do not feel a part of the process or think their feedback will not be taken seriously or
valued or teachers who do not effect changes consequent on feedback experience lower
response rates (Beran and Rokosh, 2009).
What can lecturers do to increase response rate?
The response rates are important as these evaluations are frequently used for consideration
in tenure and promotion, hiring and pay increase decisions (Hammonds et al., 2017).
Teaching
evaluation and
student
response
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
PRR
My viewpoint is that response rate can be increased if lecturers are informed about the
timing of when the surveys are sent out, so they can also make a personal appeal (both in
class and by email) to the students to complete their course evaluation surveys.
In this communication, lecturers should explain to the students how their comments
would be taken seriously, and how it will be used to improve teaching (Heinert and Roberts,
2016).
The key is to inform students about the purpose of evaluations:
Let students know that you will use their feedback to make changes in the course.
Give students some examples of useful feedback you have received in the past, and
how the course/pedagogy has benefited in response.
This best practice will show you improved results, and if you also want to score better in
these evaluations, start giving chocolate cookies to your students (ESA, 2018).
References
Anderson, H.M., Cain, J. and Bird, E. (2005), “Online student course evaluations: review of literature and
a pilot study”, American Journal of Pharmaceutical Education, Vol. 69 No. 1, p. 5.
Anderson, J., Brown, G. and Spaeth, S. (2006), “Online student evaluations and response rates
reconsidered”, Innovate: Journal of Online Education, Vol. 2 No. 6.
Avery, R.J., Bryant, W.K., Mathios, A., Kang, H. and Bell, D. (2006), “Electronic course evaluations: does
an online delivery system influence student evaluations?”, The Journal of Economic Education,
Vol. 37 No. 1, pp. 21-37.
Ballantyne, C. (2003), “Online evaluations of teaching: an examination of current practice and considerations
for the future”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 103-112.
Ballantyne, C. (2004), “Online or on paper: an examination of the differences in response and respondents
to a survey administered in two modes”, in International Conference of Australasian Evaluation
Society, Adelaide, October.
Bennett, L. and Nair, C.S. (2010), “A recipe for effective participation rates for web-based surveys”,
Assessment & Evaluation in Higher Education, Vol. 35 No. 4, pp. 357-365.
Benton, S.L., Webster, R., Gross, A.B. and Pallett, W.H. (2010), An Analysis of IDEA Student Ratings of
Instruction Using Paper versus Online Survey Methods 2002-2008 Data, The IDEA Technical Center.
Beran, T.N. and Rokosh, J.L. (2009), “Instructors’ perspectives on the utility of student ratings of
instruction”, Instructional Science, Vol. 37 No. 2, pp. 171-184.
Berk, R.A. (2012), “Top 20 strategies to increase the online response rates of student rating scales”,
International Journal of Technology in Teaching and Learning, Vol. 8 No. 2, pp. 98-107.
Crews, T.B. and Curtis, D.F. (2011), “Online course evaluations: faculty perspective and strategies for
improved response rates”, Assessment & Evaluation in Higher Education, Vol. 36 No. 7, pp. 865-878.
Dommeyer, C.J., Baum, P., Hanna, R.W. and Chapman, K.S. (2004), “Gathering faculty teaching
evaluations by in-class and online surveys: their effects on response rates and evaluations”,
Assessment & Evaluation in Higher Education, Vol. 29 No. 5, pp. 611-623.
Donovan, J., Mader, C.E. and Shinsky, J. (2006), “Constructive student feedback: online vs traditional
course evaluations”, Journal of Interactive Online Learning, Vol. 5 No. 3, pp. 283-296.
Emery, L., Head, T., Zeckoski, A., Yu. and Borkowski, E. (2008), “Deploying an open source, online
evaluation system: multiple experiences”, Presentation at Educause 2008, October 31, Orlando, FL.
ESA (European Society of Anaesthesiology) (2018), “Teachers who give cookie rewards score better in
evaluations”, ScienceDaily, 17 June, available at: www.sciencedaily.com/releases/2018/06/
180604182502.htm
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
Gaillard, F.D., Mitchell, S.P. and Kavota, V. (2011), “Students, faculty, and administrators perception of
students evaluations of faculty in higher education business schools”, Journal of College
Teaching & Learning (Tlc)), Vol. 3 No. 8.
Goodman, J., Anson, R. and Belcheir, M. (2015), “The effect of incentives and other faculty-driven
strategies to increase online student evaluation response rates”, Assessment & Evaluation in
Higher Education, Vol. 40 No. 7, pp. 958-970, doi: 10.1080/02602938.2014.960364.
Guder, F. and Malliaris, M. (2010), “Online and paper course evaluations”, American Journal of Business
Education (AJBE), Vol. 3 No. 2, pp. 131-138.
Hammonds, F., Mariano, G.J., Ammons, G. and Chambers, S. (2017), “Student evaluations of teaching:
improving teaching quality in higher education”, Perspectives: Policy & Practice in Higher
Education, Vol. 21 No. 1, pp. 26-33, doi: 10.1080/13603108.2016.1227388.
Hardy, N. (2003), “Online ratings: fact and fiction”, New Directions for Teaching and Learning, Vol. 2003
No. 96, pp. 31-38.
Hatfield, C.L. and Coyle, E.A. (2013), “Factors that influence student completion of course and faculty
evaluations”, American Journal of Pharmaceutical Education, Vol. 77 No. 2, p. 27.
Heinert, S. and Roberts, T.G. (2016), “Factors motivating students to respond to online course
evaluations in the college of agricultural and life sciences at the university of Florida”, NACTA
Journal, Vol. 60 No. 2, pp. 189-194, available at: http://eds.a.ebscohost.com.proxy1.ncu.edu
Hmieleski, K. and Champagne, M.V. (2000), “Plugging in to course evaluation”, The Technology Source.
Johnson, T. (2002), “Online student ratings: will students respond? Online student ratings of
instruction”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 49-59.
Kasiar, J.B., Schroeder, S.L. and Holstad, S.G. (2002), “Comparison of traditional and web-based course
evaluation processes in a required, team-taught pharmacotherapy course”, American Journal of
Pharmaceutical Education, Vol. 66 No. 3, pp. 268-270.
Khorsandi, M., Kobra, A., Ghobadzadeh, M., Kalantari, M. and Seifei, M. (2012), “Online vs traditional
teaching evaluation: a cross-sectional study”, Procedia – Social and Behavioral Sciences, Vol. 46
No. 2012, pp. 481-483.
Laubsch, P. (2006), “Online and in-person evaluations: a literature review and exploratory comparison”,
Journal of Online Learning and Teaching, Vol. 2 No. 2, pp. 62-73.
Layne, B.H., DeCristoforo, J.R. and McGinty, D. (1999), “Electronic versus traditional student ratings of
instruction”, Research in Higher Education, Vol. 40 No. 2, pp. 221-232.
Mau, R.R. and Opengart, R.A. (2012), “Comparing ratings: in-class (paper) vs out of class (online)
student evaluations”, Higher Education Studies, Vol. 2 No. 3, p. 55.
Miller, M.H. (2010), “Online evaluations show same results, lower response rate”, The Chronicle of
Higher Education.
Morrison, R. (2011), “A comparison of online versus traditional student end-of-course critiques in
resident courses”, Assessment & Evaluation in Higher Education, Vol. 36 No. 6, pp. 627-641.
Nair, C.S., Adams, P. and Mertova, P. (2008), “Student engagement: the key to improving survey
response rates”, Quality in Higher Education, Vol. 14 No. 3, pp. 225-232.
Nevo, D., McClean, R. and Nevo, S. (2010), “Harnessing information technology to improve the process
of students’ evaluations of teaching: an exploration of students’ critical success factors of online
evaluations”, Journal of Information Systems Education, Vol. 21 No. 1, p. 99.
Nowell, C., Gale, L.R. and Handley, B. (2010), “Assessing faculty performance using student evaluations
of teaching in an uncontrolled setting”, Assessment & Evaluation in Higher Education, Vol. 35
No. 4, pp. 463-475.
Nulty, D.D. (2008), “The adequacy of response rates to online and paper surveys: what can be
done?”, Assessment & Evaluation in Higher Education, Vol. 33 No. 3, pp. 301-314,
doi: 10.1080/02602930701293231.
Teaching
evaluation and
student
response
Downloaded by University of The West Indies at Mona At 08:45 30 October 2018 (PT)
PRR
Perrett, J.J. (2013), “Exploring graduate and undergraduate course evaluations administered on paper
and online: a case study”, Assessment & Evaluation in Higher Education, Vol. 38 No. 1, pp. 85-93.
Samuels, B. (2013), Increasing Number of Academic Departments Use Online Course Evaluations,
CampusWest, 30 April.
Sax, L.J., Gilmartin, S.K. and Bryant, A.N. (2003), “Assessing response rates and nonresponse bias in
web and paper surveys”, Research in Higher Education, Vol. 44 No. 4, pp. 409-432.
Stowell, J.R., Addison, W.E. and Smith, J.L. (2012), “Comparison of online and classroom-based
student evaluations of instruction”, Assessment & Evaluation in Higher Education, Vol. 37
No. 4, pp. 465-473.
Wode, J. and Keiser, J. (2011), “Online course evaluation literature review and findings”. A Report from
Academic Affairs, Columbia University, Chicago.
About the author
Tashfeen Ahmad expanded his understanding of Psychology at Harvard University and joined The
University of the West Indies, Mona, Jamaica, with 10 years of general management experience. He
has taught courses in International Business, Production Management, Operations Management,
Quality Management and Change Management. His research work focuses on the future of higher
education and learning technologies. Tashfeen Ahmad can be contacted at: mrtashfeen@hotmail.com
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com