Thursday, September 22, 2005

Bayesian grading

Students' scores on tests are random variables; they are not totally accurate reflections of ability. Ability sets the mean but there is normal variation around it. But given that the test scores are the only observations that the professor gets, they are also the best estimators for the students' abilities.

But tests don't occur in a vacuum: there are problem sets, discussions, office hours, etc. So shouldn't professors use some sort of Bayesian calculation when giving out final grades? This is exactly what professors who say that "problem sets don't count unless you are right between two grades" are doing. They're willing to accept the estimator if it gives the student a solid A even if they thought she was a B student. But if the estimator is not too different from their prior, if the test score is in the A-/B+ range, they want to be able to bring it down to a B+.

It's unclear whether or not this is a good practice, since the data the professor uses to form his pre-test prior might not be a good indicator of the student's true ability, even if he thinks it is.


Post a Comment

<< Home