Philip B. Stark & Richard Freishtat: What Evaluations Measure: Part II:
- Student teaching evaluation scores are highly correlated with students’ grade expectations
- Effectiveness scores and enjoyment scores are related
- Students’ ratings of instructors can be predicted from the students’ reaction to 30 seconds of silent video of the instructor: first impressions may dictate end-of-course evaluation scores, and physical attractiveness matters
- The genders and ethnicities of the instructor and student matter, as does the age of the instructor….
- Controlled, randomized experiments are the gold standard…. The only controlled randomized experiments on student teaching evaluations have found that student evaluations of teaching effectiveness are negatively associated with direct measures of effectiveness….
- The survey questions apparently most influenced by extraneous factors are exactly of the form we ask on campus: overall teaching effectiveness….
It’s time for Berkeley to revisit the wisdom of asking students to rate the overall teaching effectiveness of instructors, of considering those ratings to be a measure of actual teaching effectiveness, of reporting the ratings numerically and computing and comparing averages, and of relying on those averages for high-stakes decisions such as merit cases and promotions.
In the third installment of this blog, we discuss a pilot conducted in the Department of Statistics in 2012–2013 to augment student teaching evaluations with other sources of information. The additional sources still do not measure effectiveness directly, but they complement student teaching evaluations and provide formative feedback and touchstones. We believe that the combination paints a more complete picture of teaching and will promote better teaching in the long run.