Legal Commentary

As I noted in a post last week, some colleagues and I are reevaluating our law school's tenure standards and procedures.  We've just begun to talk about teaching evaluation.  Our current practice is to rely primarily on three sources of information: (1) student evaluation scores (derived from forms students fill out in class near the end of the semester), (2) classroom visits by tenured colleagues, and (3) written evaluations from one or two dozen randomly selected former students.  There is some dissatisfaction, though, with the weight given to the student evaluation scores.  It is sometimes argued, for instance, that students unfairly penalize rigorous, intellectually challenging teachers, while rewarding teachers who demand little and have reputations as generous graders.

I don't know if these charges are true, but I do agree that our deliberations would sometimes benefit from more information than we normally have.  In particular, I sometimes feel myself at a loss when confronted with a consistent pattern of below-average scores.  Such a pattern is surely at least a troubling a sign, but, on the other hand, it is logically impossible for everyone on a faculty to be above average.  My concerns are heightened in the rare instances when I see scores falling more than one standard deviation below the faculty mean, but, even then, there seems always to be some extenuating circumstance or another (e.g., first time teaching an especially difficult course).  Sometimes classroom visitation reports and narrative evaluations from former students help to illuminate what lies behind a pattern of below-average scores, but not always. 

As our committee begins to bat these issues around, I'm trying to think of ways that we might helpfully enrich the available information when we evaluate teaching.  My working list appears after the jump.  As in my previous post, I'd welcome reactions either in the comments or off-line.

1.  Use a mix of announced and unannounced classroom visits.  At present, we rely almost entirely on announced classroom visits, which raises concerns that our classroom visitors may be seeing unrepresentative teaching performances.

2.  Video record one or two classes per year so that all members of the Promotion and Tenure Committee can view a sampling of a candidate's classes.  Our new building has built-in recording equipment that can be used much more unobtrusively than would have been possible in our older facility.

3.  Require more classroom visits.  At present, our untenured folks normally have one class visit per year (which is conducted by two tenured faculty members who normally prepare a joint report).

4.  Consider grading data alongside student evaluation data.  If a faculty member is a demonstrably tough grader, that might help to explain poor evaluations. 

5.  Interview some former students, rather than relying solely on written statements from them.  A more informal back-and-forth exchange may generate more information and help to clarify vague comments.

6.  Collect and disseminate narrative comments from students on the end-of-semester evaluation forms.  At present, the P&T Committee normally only evaluates the numerical scores.

7.  Report and assess scores in a more nuanced way.  At present, we normally evaluate scores in comparison to the overall faculty mean.  It is possible that this disadvantages faculty members whose teaching packages are weighted towards large required courses and benefits faculty members whose teaching packages contain more small-enrollment electives.  In order to address this concern, we might report separate faculty means for different categories of classes.

8.  Evaluate the quality of student work product (papers and/or exams).  It is hard to imagine that it would be cost-benefit-justified for the P&T Committee effectively to regrade a stack of 50 exams, but perhaps an appropriate random sample could be assessed.

I should be clear that I see drawbacks to all of the items on my working list, so I don't necessarily expect to be advocating for any of them.