Badly formulated questions
Some students complained about the unclear formulation of question 20
and question 25. Here is the percentage of wrong answers as function
of the question number:
.
The question number 20 got 52% wrong answers while question number 25
got 36% wrong answers. I guess these numbers are consistent with a random
hypothesis (binomial distribution) within one sigma of 1/sqrt(19)~20%.
So I agree to remove the questions number 20 and 25 from the analysis of
the examination results. Thus only 44 of the original 46 questions will
by included.
Passed/not-passed
Under (a slightly violated) assumption that the questions are
uncorrelated, a hypothesis that a student gave random answers and
obtained k wrong answers out of n questions corresponds to a binomial
distribution Bin(n,k)/2^n. Here are the experimental results together
with the theoretical hypothesis:
.
A reasonable threshold would be 4% probability that a student answered
randomly. This threshold corresponds to 17 wrong answers. Apparently all
students have answered not randomly (with a probability larger than 96%)
with a clear bias toward correct answers. Therefore since all students have
less than 17 wrong answers and since everybody has delivered the
obligatory exercises, all students have passed the course.
Self-evaluation and the evaluation formula
Here plotted is the self-evaluation character a student has claimed as
function of the number of wrong answers they got, together with a linear
fit to the data, and a line, 12-0.5*k, from 12 at zero wrong answers
to 4 at the maximum allowed wrong answers (4 been chosen as lowest mark
for those who submitted the (allegedly a bit too big)) homework:
.
Apparently the students with the least wrong answers (the 9 left points,
some of them are multiple) are actually well correlated with the linear
estimate. The last 5 points seem to be an overestimation. So, the simplest
linear formula 12-0.5*k seems to correlate well with the expections of
the students with the least wrong answers and will then be used to mark
the exam.