Do you also find it hard to come up with good multiple choice questions? Then maybe you will like the idea of letting students propose (rather than just answer) questions. A colleague suggested this idea, arguing that it would benefit the students (creating a question requires mastering the material) and would save me work as well.

I liked this idea, and during the last three years I have applied it in my undergrad software testing course. This is a course for around 200 students which are evaluated based on an individual multiple choice exam (besides programming work conducted in pairs).

In class, I discuss example questions, and I invite students to come up with their own. The logistics are as follows:

• An exam consists of 40 multiple choice questions.
• Students can submit their questions until one week before the exam.
• As a teacher I decide which (if any) of the questions I include, and whether I think changes to the questions are necessary.
• If I include a student question, the student benefits from knowing the answer and from receiving a small bonus for submitting an included question.

To help the students in creating questions, I point them to Cem Kaner’s post on writing multiple choice test questions. I explain that for each question I need:

• A clear stem of one or two sentences that is meaningful in itself;
• One clear correct choice;
• Three distractors that are approximately equally plausible yet also objectively incorrect.

So far, I have used this procedure for eight exams during the last three years.

The students who have proposed questions that I include consistently turn out to belong to the best. This probably means that only very good students go through the effort of creating a question; It also suggests that trying to come up with a question is a good way of preparing for an exam.

For each exam I receive 10-20 questions from around five students: This very much depends on the individual students and may vary per year. Some students recognize the opportunity and submit 20 questions; But most consider it too difficult and do not come up with any.

I typically include 3-5 student questions in the exam (so one in ten questions comes from a student). This essentially depends on the number of good student questions proposed — I don’t impose an upper limit on the number of questions students can submit nor on the total number of student questions that I’m willing to include.

It is only at the exam that the students find out which questions I ask, and whether any of their questions are included. So while there is the possibility that all students share and know in advance some of the questions that might be asked, the students still need to prepare to answer other questions.

My class wondered whether I would be willing to let all 40 questions be provided by a student: My answer was ‘yes’: if a student masters the material so well that he or she is able come up with 40 usable questions covering all the material, that students deserves the highest grade.

Not all submitted questions are usable. I haven’t done the precise math, but I think I include around 20% of the proposed questions. Reasons not to include a question typically are that the question is too simple, that it is ambiguous (some distractors can be considered correct too), or that it overlaps with another question that I consider better. Some students also propose (small variations on) questions that I had used in exams of earlier years. If the similarity is too big, I reject the question.

In some cases I adopt the underlying “idea” of a proposed question, yet rewrite it substantially. In those cases the proposing student still receives the bonus point; Furthermore, the student will probably still know the correct answer.

The best part about involving students in exam creation is that some of the proposed questions are better than I could have made myself. Such questions relate to the students’ own experience (e.g.: “In an earlier course we had to aim at 80% line coverage. In light of what we learned in this course, which of the following …”).

Overall, I am very happy with this way of involving students in the exam creation process. It not only saves me some (though not much) work — it also results in inspirational questions that I could not have invented myself. And, perhaps most importantly, it makes exam creation a lot more fun to me.

Acknowledgments:

• The idea to let students propose their own questions was suggested to me by Julia Caussin, programme coordinator of the bachelor computer science at Delft University of Technology.

• Image credit: bilal-kamoon, flickr, CC BY 2.0.

See also:

• Isabel Gauthier, Teaching and Evaluating All at Once: Asking Students to Write Their Own Questions. Center for Teaching, Vanderbilt University.

• Carlos Gonzalez-Cabezas, Olivia S. Anderson, Mary C. Wright, and Margherita Fontana. Association Between Dental Student-Developed Exam Questions and Learning at Higher Cognitive Levels. Journal of Dental Education, 2015 (abstract).