In EvaluationKit, the “Detailed Report + Comments” will provide instructors with information on means (i.e., averages), medians (i.e., midpoints), as well as distributions (i.e., the percentage of students selecting each response choice). Most student ratings researchers suggest that distributions offer the most valuable lens on course feedback (e.g., Arreola, 2007; Linse, 2016; Ory, 2006). For example, looking at the percentage of students who agree or strongly agree with specific statements provides more meaningful data to interpret when compared to a numerical mean between 0 and 5.

When accessing your own feedback, first, take a look at where the majority of your scores are located (Ory, 2006). For each item, if a large percentage of students are in agreement (i.e., percent strongly and somewhat agree), this is a signal that, in general, the student experience of that aspect of the course was positive. On the other hand, because most faculty nationwide receive ratings that lean toward the upper end of the scale, many scores of “strongly disagree,” “somewhat disagree” or “neutral” are a signal that students perceived significant challenges with the course or instructor (Arreola, 2007). In this scenario, see “How can I improve student feedback in future terms?” below.

Instructors may also wish to take note of contextual factors associated with the class, e.g., size, level, or student-reported reasons for taking the course. Instructors of elective courses, graduate courses, and small classes tend to receive higher ratings, although the impact tends to be small (Arreola, 2007; Benton & Cashin, 2012; Linse, 2016). Additionally, although the literature is quite complex and evolving, studies have examined how interactional factors involving instructor identity have some impact on ratings, including career stage, identity (or perceived identity) in relationship to course content, and alignment between a student’s identity and their perception of the instructor’s identity (e.g., Basow & Martin, 2012; Linse; Mengel, Sauermann, Zölitz, 2018). 

Many instructors find comments to be the most helpful component of student feedback because the narrative can offer specific actionable suggestions – or affirm what students report is working well for their learning. Accordingly, the new Brown course feedback system offers more options for students to comment on specific aspects of a course. Examples of these questions include, “What specific advice would you have for this instructor about changes that would enhance your learning?” and “What would you like to say about this course to a student who is considering taking it in the future?”

However, comments can also be frustrating, especially because contradictory comments are common (Linse, 2016). Additionally, compared with quantitative feedback, one drawback with comments is that it is easier to focus on negative issues or isolated student experiences. This is particularly important for comments given to female faculty and faculty of color because “we often see students with biased views represented in the tails of the distribution” (Linse, p. 9).

If there are few comments, it may be helpful to encourage future classes to offer narrative feedback, giving examples of comments that have been useful to your teaching in the past.

Helpful strategies for interpreting comments include:

• Ask someone else (e.g., a colleague, a Sheridan staff member) to read through your comments to identify patterns. Often, external readers can more easily see trends, rather than focus on negative comments or isolated feedback, and offer suggestions based on these patterns.
• Group comments into themes and count how many comments are associated with each theme (Linse, 2016). Higher frequency themes should receive more attention.

Global items (e.g., “Overall, I rate this course as effective” and “Overall, I rate this instructor as effective”) often draw the most attention but can be difficult to interpret. If these are low, check the questions that ask for feedback on clarity (“The instructor made course material clear…”) and organization (“The instructor was well prepared for each class…”). These dimensions tend to be the best ways to explain variation in global questions (Feldman, 2007). 

Educational researcher Harry Murray (2007) studied observable teaching behaviors that are associated with these two dimensions of feedback. His research suggests that to improve students’ sense of clarity, small changes to consider include:

• Writing key terms on the board
• Giving multiple examples of a concept
• Pointing out practical applications of a concept

Teaching behaviors that are associated with student perceptions of organization include (Murray):

• Providing an outline or agenda for each class
• Noting when you are transitioning from one topic to another
• Noting sequence and connections, or highlighting how each course topic fits into the class as a whole
• Periodically stopping to review (or ask students to review) key course topics

Finally, a number of studies suggest that collecting early student feedback has a positive impact on end-of-term ratings, and meeting with a consultant to discuss the feedback has an even greater impact (Cohen, 1980; Finelli, et al., 2008; Marsh, 2007; Penny & Coe, 2004). The Sheridan Center offers an early feedback service that combines a discussion with the faculty member, an observation, and a brief mediated focus group discussion with students. 

Early Student Feedback Consultations

Even a few returned course feedback forms may offer valuable insights into student experience of a course, but generally, the higher the response rate, the more assurance you can have that your quantitative feedback is consistent. Recommended response rates vary by size of the course: 

(Franklin, 2001)

EvaluationKit enables you to monitor response rates on an ongoing basis, so if your response rates seem particularly low, an email reminder can be helpful.

What can you do if your response rates are low this term but you seek to increase them in the future? Helpful evidence-based approaches to use in future terms include describing to students how you have changed the course based on prior student feedback (Ory, 2006). Additionally, response rates tend to improve if instructors ask students to bring a mobile device at the end of the term and provide time for them to complete the feedback at the beginning of class (Goos & Salomons, 2017). One Brown Chemistry senior adds, “An instructor can influence you to send feedback by simply giving time to fill it out. Most students do not believe there is an extra 15 minutes in a day (we know logically there is, but it’s hard to motivate) and so giving the time in class forces you to do it.” 

Teaching is multidimensional, meaning that a number of activities form the foundation of excellent instruction (Theall & Arreola, 2006). Although student course feedback offers valuable student-driven input on some of these, students are not the best source of information for other dimensions, such as content expertise. Therefore, instructors who are writing a teaching narrative or putting together a teaching portfolio will wish to complement the course feedback with other perspectives. Some possibilities are offered below.

While these data do not have as much research behind them about questions of reliability, validity, or bias, a multimodal approach offers important information about other components of effective teaching (Linse, 2016).

Additionally, for any personnel-related process like a job search or review, it is important to provide (when possible) the history of scores over time, in order for a reviewer to identify patterns (Franklin, 2001). To allow for this historical presentation, the new feedback form retains identical wording of two key global items (“Overall, this is an effective course” and “Overall, this is an effective instructor”).

Generally, comparisons of ratings – such as with a department average or another instructor – are to be avoided because of the possibility of drawing erroneous conclusions. However, if an instructor needs to present such a comparison, (Franklin, 2001) offers a step-by-step guide to statistical calculations that offer a more accurate comparative approach (e.g., confidence intervals).

Even with this comparison, the most important focus should be on the descriptor associated with the numerical score, rather than small differences in means. For example, although an average score of 4.5 is higher than 4.0, both scores correspond to an “agree” response – which is the descriptor students used when they filled out the survey (Dewar, 2017).