The Case Against Multiple Choice Exams
September 26, 2007
Originally posted on: September 26th, 2007
Updated on: November 27th, 2007
One thing that has greatly disappointed me about college undergrad courses is the reliance many professors place on multiple choice (MC) exams. This semester, all five of the classes I'm taking have used multiple choice questions on exams. This is unfortunate because, as I will attempt to make the case, multiple choice exams have many flaws and are a poor metric to determine a student's knowledge on a given subject. Many of the arguments here have been borrowed from the National Center for Fair and Open Testing and examples are from my own experiences with multiple choice testing.
Below are my objections to multiple choice exams:
MC exam results are mathematically skewed - say I sit down at a 500 question multiple choice exam for a class I never attended, never read the textbook, and know nothing about: when I receive the grade for my exam, I'll have received a score of 25% (within some standard deviation) if I filled in all the bubbles randomly. Clearly, I do not understand 25% of the material, I understand 0% of it. My score significantly misrepresents my knowledge on the topic. Of course, this only works if all bubbles are filled in randomly, but there are other mathematical problems with multiple choice...
MC exam questions are all or nothing - since every question only has one right answer, I can receive 0% (completely failure) for a question, even if I have some amount of knowledge on the subject. Consider the following example question: Q: Was the infantry invasion of Japan a viable alternative to the use of the atomic bomb to end World War II? Is so, why? If not, why not?
A. Yes; transport ships were available in sufficient numbers.
B. Yes; island defenses in Japan were minimal.
C. No; estimated casualties would have been much greater.
D. No; Japan was on the verge of having an atomic bomb.
Lets say I think the answer is "No" and I can make a great case to justify that answer. Perhaps I would argue that Japan's military was more powerful than the U.S. believed and that our military would have lost in an infantry invasion. Unfortunately, that isn't a choice; and now I am forced to guess between answers C and D. The "wanted answer" in this example is C. Had I guessed D, I would have received 0% credit and failed the question, even though I had a strong grasp of the subject; statistically, I have a 50% chance of getting the answer correct, which is unfair if I understand, say, 80% of the subject.
Mutiple choice exams often employ misleading or poorly worded questions - rather than ask a question and allow the student to answer it as he/she chooses, MC exams force students to try to decipher the meaning of a question and think "what does the professor want me to say". Consider this example from a recent psychology exam I took: Q: Smoking during pregnancy is most likely to cause which of the following birth defects?
A. Low birth weight
B. Autism
C. General Anxiety
D. Smoking will cause many birth defects
I look at this question and think... "well, the most common birth defect from smoking is low birth weight, BUT I know that smoking causes other birth defects as well." So now I'm torn over whether I should pick A or D. I know that smoking causes a number of birth defects, including preterm birth, cerebral palsy, mental retardation, learning problems, withdrawal symptoms, etc. So even though I know more about this subject than the question is asking, my goal is to decipher what the professor means by the terms "most" and "many". Some of my professors in the past have used "most" to refer to one or more correct answer, but this particular professor wanted it to mean "the number one statistical birth defect, excluding the number two, three, etc. defects". I have also had professors who use the word "many" as an inclusive term; so to me, so I interpret choice D as "low birth weight, preterm birth, cerebral palsy, etc." I chose answer D and received 0% (complete failure) for the question because I thought too deeply into the question and misjudged the intent of the professor. Had this been a short answer question, I could have identified that low birth weight is the statistically most common birth defect, but included that there is still a risk of developing other problems. Since it was multiple choice, I failed the question and have no way to prove what I was thinking at the time.
Multiple choice exams discourage learning outside of a narrow curriculum - when studying for multiple choice exams, I often find myself memorizing vocabulary terms or cramming in concepts I know will be on the exam. Rarely do I look at the big picture, and often I forget the material hours or days after the exam ends. The very nature of MC exams discourages learning broad concepts because you are often punished (as in the above example) for knowing too much. On a multiple choice economics exam, for example, it is more important to memorize the metrics used to measure GDP than it is to know the connections between GDP and other important aspects of economics. Critical thinking is discouraged; mindless memorization is the priority.
The concept of a "best answer" makes MC questions confusing and frustrating - MC questions often have many choices that could be "plausible" answers but they force students to choose the "best answer." This is problematic because "best" answer is a vague concept and because choosing the "second best" answer might still demonstrate some understanding of a concept. Unfortunately, since multiple choice is all or nothing, this often creates extreme frustration for students. Consider this example from a recent philosophy exam I took: Which of the following can be known using a posteriori knowledge (a posteriori = empirical evidence or use of human senses):
A. Triangle have three sides
B. Martians live on Mars
C. All bachelors are unmarried
D. Descartes wrote Meditations on First Philosophy
At first glance I choose B, because obviously if there was life on Mars I would have to see pictures with my eyes or hear about it from someplace. But wait, choice D also seems correct, because in order to know what Descartes authored I'd have to read his works or learn about it from someplace else. So whats going on? Apparently there are two correct answers here. I pick B and am wrong. The professor says the answer is D because Descartes really did write Meditations on First Philosophy but there is no known life on Mars. After trying to argue my case I'm told that even though B might be a plausible answer, D is the "best" answer and that I should have picked D. Forget the fact that B is a plausible answer and forget the fact that I understand the concept of a posteriori knowledge and can justify my answer... I get 0% and fail the question.
"All of the above" and "none of the above" remove the "choice" from multiple choice - my Psychology professor justifies multiple choice exams by stating that they are easier than any other exam formats because they require the tester only to "recognize" the correct answer from a list of choices rather than recall some information. To understand this theory, think of it this way: say you've watched a few football games in your life but know virtually nothing about the game. If someone asks you what it is called when a player runs the ball into the end zone, you would probably have no idea. But what if you could choose between the following choices: run, basket, touchdown, goal. Ah ha! Now you remember that its called a touchdown. So isn't that proof that multiple choice is easier? Maybe for very basic questions, but not when you throw in "all of the above" or "none of the above" answers into the mix. Consider this example from the same professor's exam: If you occasionally attend a rock concert without wearing ear protection, chances are good that you would suffer:
A. Total hearing loss after a few years
B. Damage to hair cells
C. Damage to eardrum
D. All of the above
This question suffers from many of the downfalls of multiple choice exams. First, it uses many misleading words, like "occasionally", "good" and "few". Second, it relies on a "best answer" since A, B and C are all theoretically plausible. And finally, it offers a choice to choose "all of the above." This question goes way beyond simply recognizing the correct answer, it can only be solved by playing mind games and trying to figure out what the professor wants you to say. I pick choice D, figuring that since all of the above are plausible answers, chances would be "good" that all of them would occur. Plus I figure that since the D choice is inclusive of the best choice, it would then become the best choice. I'm wrong (again) since there isn't a "good" chance of damaging the eardrum or developing total hearing loss (even after a "few" years) and therefore the answer which includes those choices is also wrong.
True/False exams are multiple choice in disguise - some people believe that true/false is the easiest style of testing because for every question there is a 50% chance of getting it correct, even without knowing the material! Fair enough; but when you "do" know the material, these questions can be a royal pain, since its often possible to justify both the true and the false choices in your head. Take this example from a recent management exam I took: True or false: honesty is absolutely essential to leadership? The obvious answer to the question would be "true" but for a few seconds I might contemplate a scenario where honesty would not be essential to leadership. After all, if there is just one exception to that statement then the "false" answer would seem plausible, making this a matter of picking which is better, a tricky task.
Multiple Choice is not more "objective" than other exam formats when applied to a single class - one of the arguments some people use to justify multiple choice exams is that they can be graded more objectively than other forms of testing. For example, on the SAT, the essay section is graded by hundreds of different people across the country... in theory your score could be completely different depending on who is reading your exam. You might write a very similar essay to the person sitting next to you during the exam but she might get great marks and you might not do well at all. On the other hand, multiple choice, which is graded by a machine, can't succumb to such subjectivity. While I give some credence to this argument for huge nation-wide standardized tests, the argument fails to hold up when applied to exams in individual classes. Since one professor will be grading all the exams, there is little risk of grading discrepancy between individual students.
So with all the problems multiple choice exams have, why do professors use them? One reason I already noted above, they have the faulty belief that multiple choice relies on "recognition" rather than "recall" and therefore are easier for students; but this line of thought seems fairly rare. In many instances, it turns out that professors use multiple choice because students actually prefer them to any other test format. How can this be? I thought about it for a few days and asked people who prefer multiple choice exams and came up with a theory.
All of the reasons I am opposed to multiple choice exams are predicated on the assumption that I understand the material and that multiple choice exams hinder me from demonstrating what I know. If, however, I didn't know the material, or didn't know it well, then I might be led to believe that I'd be better off with a multiple choice exam. After all, with multiple choice exams, at least there would be a chance I could "recognize" something and at least I have something to pick from. On a short answer exam, if I don't know the material I'll end up sitting there staring at blank sheets of paper for the entire exam period. Psychologically I think being able to pick an answer to a bunch of questions (even if the answers are wrong) is more comforting than having a bunch a bunch of blank spaces. A 50% is a 50%, regardless of whether you filled in all the bubbles and missed half or if you had a bunch of blank space on a short answer test. But while taking the test, the student will feel more confident filling in the bubbles than anything else.
Does this mean professors should still use multiple choice if a lot of students prefer it? I don't think so. The ultimate goal of any professor should be to teach as much useful material as possible and the ultimate goal of students should be to learn as much as possible. As I hope I've demonstrated, multiple choice exams do a poor job helping students learn important material but instead encourage them to "beat the test". If professors truly care about teaching students material that they can use in the future, they should shred their multiple choice exams and not look back.
After all this negativity about multiple choice, do I have a solution? Well, nothing is perfect and no one will ever be fully satisfied, but there are two exam styles which are both fair and encourage students to learn and retain material. The first consists of a series of short answer questions where students are instructed to answer some percentage of them. For example, the professor could give 5 questions and ask students to answer 4 or give 10 questions and ask students to answer 7. This style of exam ensures that students have a fair opportunity to demonstrate what they know about a topic and also gives students a confidence boost, knowing that they only have to answer those questions they know the most about.
The second style of testing is rare but seems both fair and reasonable. The professor will hand out several short answer questions 3-7 days before an exam, and 70%-80% of them will appear on the actual exam. This guarantees that students study the concepts that professors deem to be most important; the element of randomness ensures that the student prepares for all the questions, since they don't know which will be on the exam, preparing for only some would be nothing more than a roll of the dice; and having the questions beforehand gives students confidence and relieves some test anxiety.
I think if more students had the opportunity to experience these types of exams they would become increasingly negative on multiple choice. I think multiple choice could seem attractive when compared to short answer exams where students have to answer every single question; but I think the two exam styles I've proposed can solve many of the problems students currently face. Ultimately, nothing will be perfect, but we can do a better job of making sure that expensive college courses are worth it - alternatives to multiple choice exams can help.
Updated on: November 27th, 2007
One thing that has greatly disappointed me about college undergrad courses is the reliance many professors place on multiple choice (MC) exams. This semester, all five of the classes I'm taking have used multiple choice questions on exams. This is unfortunate because, as I will attempt to make the case, multiple choice exams have many flaws and are a poor metric to determine a student's knowledge on a given subject. Many of the arguments here have been borrowed from the National Center for Fair and Open Testing and examples are from my own experiences with multiple choice testing.
Below are my objections to multiple choice exams:
MC exam results are mathematically skewed - say I sit down at a 500 question multiple choice exam for a class I never attended, never read the textbook, and know nothing about: when I receive the grade for my exam, I'll have received a score of 25% (within some standard deviation) if I filled in all the bubbles randomly. Clearly, I do not understand 25% of the material, I understand 0% of it. My score significantly misrepresents my knowledge on the topic. Of course, this only works if all bubbles are filled in randomly, but there are other mathematical problems with multiple choice...
MC exam questions are all or nothing - since every question only has one right answer, I can receive 0% (completely failure) for a question, even if I have some amount of knowledge on the subject. Consider the following example question: Q: Was the infantry invasion of Japan a viable alternative to the use of the atomic bomb to end World War II? Is so, why? If not, why not?
A. Yes; transport ships were available in sufficient numbers.
B. Yes; island defenses in Japan were minimal.
C. No; estimated casualties would have been much greater.
D. No; Japan was on the verge of having an atomic bomb.
Lets say I think the answer is "No" and I can make a great case to justify that answer. Perhaps I would argue that Japan's military was more powerful than the U.S. believed and that our military would have lost in an infantry invasion. Unfortunately, that isn't a choice; and now I am forced to guess between answers C and D. The "wanted answer" in this example is C. Had I guessed D, I would have received 0% credit and failed the question, even though I had a strong grasp of the subject; statistically, I have a 50% chance of getting the answer correct, which is unfair if I understand, say, 80% of the subject.
Mutiple choice exams often employ misleading or poorly worded questions - rather than ask a question and allow the student to answer it as he/she chooses, MC exams force students to try to decipher the meaning of a question and think "what does the professor want me to say". Consider this example from a recent psychology exam I took: Q: Smoking during pregnancy is most likely to cause which of the following birth defects?
A. Low birth weight
B. Autism
C. General Anxiety
D. Smoking will cause many birth defects
I look at this question and think... "well, the most common birth defect from smoking is low birth weight, BUT I know that smoking causes other birth defects as well." So now I'm torn over whether I should pick A or D. I know that smoking causes a number of birth defects, including preterm birth, cerebral palsy, mental retardation, learning problems, withdrawal symptoms, etc. So even though I know more about this subject than the question is asking, my goal is to decipher what the professor means by the terms "most" and "many". Some of my professors in the past have used "most" to refer to one or more correct answer, but this particular professor wanted it to mean "the number one statistical birth defect, excluding the number two, three, etc. defects". I have also had professors who use the word "many" as an inclusive term; so to me, so I interpret choice D as "low birth weight, preterm birth, cerebral palsy, etc." I chose answer D and received 0% (complete failure) for the question because I thought too deeply into the question and misjudged the intent of the professor. Had this been a short answer question, I could have identified that low birth weight is the statistically most common birth defect, but included that there is still a risk of developing other problems. Since it was multiple choice, I failed the question and have no way to prove what I was thinking at the time.
Multiple choice exams discourage learning outside of a narrow curriculum - when studying for multiple choice exams, I often find myself memorizing vocabulary terms or cramming in concepts I know will be on the exam. Rarely do I look at the big picture, and often I forget the material hours or days after the exam ends. The very nature of MC exams discourages learning broad concepts because you are often punished (as in the above example) for knowing too much. On a multiple choice economics exam, for example, it is more important to memorize the metrics used to measure GDP than it is to know the connections between GDP and other important aspects of economics. Critical thinking is discouraged; mindless memorization is the priority.
The concept of a "best answer" makes MC questions confusing and frustrating - MC questions often have many choices that could be "plausible" answers but they force students to choose the "best answer." This is problematic because "best" answer is a vague concept and because choosing the "second best" answer might still demonstrate some understanding of a concept. Unfortunately, since multiple choice is all or nothing, this often creates extreme frustration for students. Consider this example from a recent philosophy exam I took: Which of the following can be known using a posteriori knowledge (a posteriori = empirical evidence or use of human senses):
A. Triangle have three sides
B. Martians live on Mars
C. All bachelors are unmarried
D. Descartes wrote Meditations on First Philosophy
At first glance I choose B, because obviously if there was life on Mars I would have to see pictures with my eyes or hear about it from someplace. But wait, choice D also seems correct, because in order to know what Descartes authored I'd have to read his works or learn about it from someplace else. So whats going on? Apparently there are two correct answers here. I pick B and am wrong. The professor says the answer is D because Descartes really did write Meditations on First Philosophy but there is no known life on Mars. After trying to argue my case I'm told that even though B might be a plausible answer, D is the "best" answer and that I should have picked D. Forget the fact that B is a plausible answer and forget the fact that I understand the concept of a posteriori knowledge and can justify my answer... I get 0% and fail the question.
"All of the above" and "none of the above" remove the "choice" from multiple choice - my Psychology professor justifies multiple choice exams by stating that they are easier than any other exam formats because they require the tester only to "recognize" the correct answer from a list of choices rather than recall some information. To understand this theory, think of it this way: say you've watched a few football games in your life but know virtually nothing about the game. If someone asks you what it is called when a player runs the ball into the end zone, you would probably have no idea. But what if you could choose between the following choices: run, basket, touchdown, goal. Ah ha! Now you remember that its called a touchdown. So isn't that proof that multiple choice is easier? Maybe for very basic questions, but not when you throw in "all of the above" or "none of the above" answers into the mix. Consider this example from the same professor's exam: If you occasionally attend a rock concert without wearing ear protection, chances are good that you would suffer:
A. Total hearing loss after a few years
B. Damage to hair cells
C. Damage to eardrum
D. All of the above
This question suffers from many of the downfalls of multiple choice exams. First, it uses many misleading words, like "occasionally", "good" and "few". Second, it relies on a "best answer" since A, B and C are all theoretically plausible. And finally, it offers a choice to choose "all of the above." This question goes way beyond simply recognizing the correct answer, it can only be solved by playing mind games and trying to figure out what the professor wants you to say. I pick choice D, figuring that since all of the above are plausible answers, chances would be "good" that all of them would occur. Plus I figure that since the D choice is inclusive of the best choice, it would then become the best choice. I'm wrong (again) since there isn't a "good" chance of damaging the eardrum or developing total hearing loss (even after a "few" years) and therefore the answer which includes those choices is also wrong.
True/False exams are multiple choice in disguise - some people believe that true/false is the easiest style of testing because for every question there is a 50% chance of getting it correct, even without knowing the material! Fair enough; but when you "do" know the material, these questions can be a royal pain, since its often possible to justify both the true and the false choices in your head. Take this example from a recent management exam I took: True or false: honesty is absolutely essential to leadership? The obvious answer to the question would be "true" but for a few seconds I might contemplate a scenario where honesty would not be essential to leadership. After all, if there is just one exception to that statement then the "false" answer would seem plausible, making this a matter of picking which is better, a tricky task.
Multiple Choice is not more "objective" than other exam formats when applied to a single class - one of the arguments some people use to justify multiple choice exams is that they can be graded more objectively than other forms of testing. For example, on the SAT, the essay section is graded by hundreds of different people across the country... in theory your score could be completely different depending on who is reading your exam. You might write a very similar essay to the person sitting next to you during the exam but she might get great marks and you might not do well at all. On the other hand, multiple choice, which is graded by a machine, can't succumb to such subjectivity. While I give some credence to this argument for huge nation-wide standardized tests, the argument fails to hold up when applied to exams in individual classes. Since one professor will be grading all the exams, there is little risk of grading discrepancy between individual students.
***************************
So with all the problems multiple choice exams have, why do professors use them? One reason I already noted above, they have the faulty belief that multiple choice relies on "recognition" rather than "recall" and therefore are easier for students; but this line of thought seems fairly rare. In many instances, it turns out that professors use multiple choice because students actually prefer them to any other test format. How can this be? I thought about it for a few days and asked people who prefer multiple choice exams and came up with a theory.
All of the reasons I am opposed to multiple choice exams are predicated on the assumption that I understand the material and that multiple choice exams hinder me from demonstrating what I know. If, however, I didn't know the material, or didn't know it well, then I might be led to believe that I'd be better off with a multiple choice exam. After all, with multiple choice exams, at least there would be a chance I could "recognize" something and at least I have something to pick from. On a short answer exam, if I don't know the material I'll end up sitting there staring at blank sheets of paper for the entire exam period. Psychologically I think being able to pick an answer to a bunch of questions (even if the answers are wrong) is more comforting than having a bunch a bunch of blank spaces. A 50% is a 50%, regardless of whether you filled in all the bubbles and missed half or if you had a bunch of blank space on a short answer test. But while taking the test, the student will feel more confident filling in the bubbles than anything else.
Does this mean professors should still use multiple choice if a lot of students prefer it? I don't think so. The ultimate goal of any professor should be to teach as much useful material as possible and the ultimate goal of students should be to learn as much as possible. As I hope I've demonstrated, multiple choice exams do a poor job helping students learn important material but instead encourage them to "beat the test". If professors truly care about teaching students material that they can use in the future, they should shred their multiple choice exams and not look back.
After all this negativity about multiple choice, do I have a solution? Well, nothing is perfect and no one will ever be fully satisfied, but there are two exam styles which are both fair and encourage students to learn and retain material. The first consists of a series of short answer questions where students are instructed to answer some percentage of them. For example, the professor could give 5 questions and ask students to answer 4 or give 10 questions and ask students to answer 7. This style of exam ensures that students have a fair opportunity to demonstrate what they know about a topic and also gives students a confidence boost, knowing that they only have to answer those questions they know the most about.
The second style of testing is rare but seems both fair and reasonable. The professor will hand out several short answer questions 3-7 days before an exam, and 70%-80% of them will appear on the actual exam. This guarantees that students study the concepts that professors deem to be most important; the element of randomness ensures that the student prepares for all the questions, since they don't know which will be on the exam, preparing for only some would be nothing more than a roll of the dice; and having the questions beforehand gives students confidence and relieves some test anxiety.
I think if more students had the opportunity to experience these types of exams they would become increasingly negative on multiple choice. I think multiple choice could seem attractive when compared to short answer exams where students have to answer every single question; but I think the two exam styles I've proposed can solve many of the problems students currently face. Ultimately, nothing will be perfect, but we can do a better job of making sure that expensive college courses are worth it - alternatives to multiple choice exams can help.
I could not agree more with you. I am completely against them. I think it is unfair for good students and an easy way out for the education system to grade. It does not push students to study but for certain answers and then they quickly forget..
Love your post.
An image that illustrates multiple choice
We had tremendously long multiple choice tests (also matching & fill in the blank) years ago in my 100 series geography classes--and even some 200 level courses. I thought it worked pretty well where a key aim of the course was to master a body of data. However, these were not just any multiple choice tests. First, it was possible for the proper answer to be some combination of the possible answers, all of them or none of them. I'm not saying that there was an option like "d. none of the above" or "c. a and b". No, there would be four or five responses that had different content and it was up to the student to put down "a & e" or "none", etc. Likewise with matching, you couldn't use process of elimination because any answer could be used multiple times and others might not be used at all.
The tests were challenging, and, frankly, a lot of fun. Moreover, in the course of 400 questions or so, you would get an occasional tension releaser. For example, matching "pink flamingos" with "tacky."
The tests cut down on students' tendencies toward BS and made giving (and grading) long tests that covered a lot of material more manageable. And while you are correct that such tests can be badly worded, a teacher who can't word a multiple choice question correctly is not a good candidate for being able to grade an essay.
Your real case is about bad teachers. Not a lot of disagreement there.
I've never heard a prof defend MC exams for pedagogical reasons. Profs give MC exams for pragmatic reasons. Short answer exams in large classes take soooooooo much longer to grade, and profs don't have time--either b/c the exam is a final and the school's grading deadlines are too tight, or the exam is a midterm and profs can't spend hour over hour grading short answer/essays and still have enough time to publish/ do service (bs committee work).
I do agree w/ the thrust of your post: any prof w/ a class of 25 students who gives an MC exam instead of an essay/short answer is cheating those students. But if you think a prof can grade 200 short answer exams and still honor his/her other professional commitments, you should try it.
Why I give multiple-choice exams:
150 students taking the test (on frequent occasions, more students that that; my maximum has been 350).
No assistance grading the test.
Time is a constraint.
Your argument is unanswered by the professors because they use MC tests for one reason only, to decrease their workload. Increasingly, it is becoming obvious that university/college lecturers are able to get away with less and less commitments toward undergrads, one of which is marking work. Multiple choice can be rifled through a scanner/computer.
Saying this, just one of the examiners in my first year science degree in Bristol, UK, had a fairer method of multiple choice, in which it was possible to achieve 0% even when ansering all questions. This was because he gave six possible answers, two of which would subract either 1 or 2 marks from your score were you to choose them. The positive options were either 1 or 2 marks, and there was a neutral option.
I felt this to be both testing and fair. If you had not studied, 25% would be a good score.
On the same course, it was a general rule that lecturers were not aloud to dedicate more than 50% of the exam to MC questions, although in some of them I think there were just MC questions, most of which were just of the a)b)c)d) type.
"So with all the problems multiple choice exams have, why do professors use them? "
- time
It takes long to grade non mc tests.
The unobvious solutions
- a staff of graders
- better designed tests
As a lucky guesser but lazy student (recent graduate actually), I was all in favour of MC tests. They let those of us who would rather waste our lives on drugs, sex, violence and insanity still graduate with advanced degrees. M to da C, fo' lyfe.
Mayte Gorostiaga, thank you for sharing the cartoon!
AgathaX, great feedback. The MC choice exam you describe sounds very much like a game to me; and I have to admit, I think it is rare to hear anyone call an exam 'fun'. Nevertheless, it would be wonderful if education was such that it wouldn't be surprising to call exams 'fun'. You are correct though, I am mostly unhappy about bad teachers. However, in my experience, bad teachers and MC exams have been positively correlated.
Anonymous #1, I understand the concern about time. I realize that some professors have other responsibilities. In my experience, though, there graduate students or other assistants available to help grade assignments or tests. Whether or not they are always available or even that helpful - I honestly do not know.
doc, point taken. I go to a small school with small class sizes and I often forget that there are lectures with a hundred or more students in them. At my school, even professors who teach multiple sections rarely have more than 100 total students.
Ian, you raise some interesting points, and it makes me wonder... how problematic is it that professors are increasingly taking on work loads that takes away from their commitment to undergrad classes? At what point does an incredibly smart professor become less helpful because he/she is unable to give enough of their time to students?
Anonymous Coward, I agree - those are obvious solutions. My beef with MC exams might be much less if the exams themselves weren't so frustrating.
Anonymous #2, thanks for sharing; I had a feeling that was the case with, as you call them, “lazy students”.
AgathaX, I think what you described is a disguised true/false test. I've heard it described as 'multiple multiple choice', but it's basically a true/fase question for each option, is it not?
I don't mind well-written multiple choice tests I particularly like them when the teacher then goes over them and lets students defend their choices. This could be done on short answer tests (and should be done on essay tests) but in those cases might require time for each student's defence, whereas when I had professors let us argue our multiple choice answers, there were only $number_of_options to be argued for each question, not the $number_of_students answers per question possible with short answer.
Even this, though, is somewhat infeasible with a 200-student course. Perhaps smaller classes is the answer, albeit an expensive one.
I'm w/anonymous #1 and doc. I refused to give MC exams when I taught sections of 50; now I teach 500 with 3 masters-level TAs that I barely trust to grade the 2-3 short-answer questions I give, let alone essays. It pains me but I have no option but to make the bulk of my exams MC. As you point out, writing "good" MC questions is hard but I'm intrigued by the formats that agathax and Ian describe. Just keep in mind that not all profs who give MC exams are doing it to torture you or to shirk their responsibilities - at many (large, often state-run) institutions, we're playing with the hand/class size we're dealt.
MC is just another teaching tool that can be used or abused.
The site below uses a multiple-choice format not as a testing but as a teaching tool -- to teach grammar, usage, spelling and journalistic style:
http://newsroom101.com
Many of the correct answers are accompanied by feedback on the grammatical point in question.
Ignoring the grading aspect, short answer exams are almost always better in my opinion because they allow students to better demonstrate their knowledge (or lack thereof). In my experience as a TA, short answer exams are much more time intensive to grade, mostly because of poor handwriting and loads of BS. They also facilitate dishonest students who would add to or change their answers once the exam is passed back, and then ask for a regrade. With multiple choice exams, it's much easier to put a red 'X' on the incorrect answer and preempt any opportunity to cheat. Multiple choice exams do add variation to your final score, but so does subjective grading on short answer questions. I try to be as fair as possible when grading, but it's very easy for graduate students, who have many other responsibilities that are far more important to their success than grading, to slack off and grade poorly and inconsistently. It'd be great for students if a professor's only responsibility was to be a good instructor and to design fair exams, but they have many other responsibilities, especially at research universities where teaching is a distant second to publishing in top journals. Oftentimes, the system is more to blame than the professor.
The universal trend is to increase teaching loads. Where once an instructor had maybe 25 in a class, now they are expected to teach 50, 75 or more. Then with three or more classes - you are looking at 200 or more tests to grade. Unless they have a team of super grading assistants or very infrequent tests - MC is necessary.
Why else do you think it is a trend? Instructors are under ever greater pressure to publish reserch, get grants etc...
Something had to give.
Oops.
"research" not "reserch"
Most of your problems are due to badly written test questions. My mother, an ex-teacher, has always said that every test question used in a test needs to be pre-tested with students of known ability to see if the smarter/better-informed kids will read any complexities into them that the author didn't expect.
She also thinks this is true of short answer questions as well. Not sure if she applies this to essay questions.
Multi-choice tests may be easy to mark, but if they are going to be decent tests they aren't easy to write.
I have to say that I completely disagree, though for different reasons than you will expect. In my opinion, ALL tests and grades do little to actually TEACH you anything. Yes, we have to track learning somehow (blah blah blah), but I would think that professors would want to spend less time reading strange attempts at cogent short answers than making better lesson plans. Therefore, a multiple choice test, taking less effort to grade, will enhance learning. It may decrease the accuracy of grading, but I think that we have to recognize that all tests are going to do a pretty poor job of this, so whatever.
LKBM, I think the ability to defend an answer would eliminate a lot of the ambiguity that makes MC exams so frustrating to begin with. Unfortunately, some professors to refuse to back down, no matter how strong of a case can be made.
Jennifer Imazeki, point taken. I am particularly bothered, however, with professors who teach advanced classes with few students, and still insist on using MC exams.
Gerald Grow, thanks for the link. Someone else was just telling me about how she uses MC as a teaching tool rather than an exam format.
Matt & PJ, thanks for the insights. I too question whether the system is set up in the best way to encourage learning and help students advance.
Tracy W, from my experience, professors often have a difficult time writing multiple-choice exams; and the result is misleading and poorly worded questions. Perhaps they would have been just as bad at writing questions for another exam format? I'll never know.
spanish bombs, you are probably correct in theory, but I wonder if it true in practice. Assuming there is a finite amount of time in a person's day, time spent grading exams is time spent not preparing lectures, etc. Fair enough. However, if there are only 2 exams per semester, and one of them is a final that occurs after all the lectures have been given, the idea that a professor would do a poor job preparing for his/her classes seems questionable.
One thing to consider, a single grader can vary on how they grade short answers depending on when they grade it. Anchoring heuristics are a problem there. If the last SA got a 3/3, the next one is more likely to as well. Same with a 0/3. Or if multiple TAs are grading, their differing criteria are a problem too.
At least, MC's are objective.
Your essay was very interesting and well-argued. However, if you look at the examples that you gave, I would like to concur that the problem was mainly with poorly-worded questions, not with the multiple-choice format itself.
Over here in Germany, multiple-choice is almost unheard of. Nearly 100% of the exams fall into the short-answer / essay format (and no, you normally cannot choose to answer only a certain part of the questions and you do not know any of the questions in advance). Having spent a while in the US, though, I know how multiple-choice exams can be like.
Having that comparison, though, makes me believe even more strongly that the problem is with bad questions and/or lazy professors, not with the format itself. Consider the following question on a short-answer business exam I took: "Name three factors which can significantly influence xyz." Seems fair, right? I had a grasp of the concept and was able to produce a coherent argument for three factors. I still got zero points on the question. When I asked the professor what was up with that, he openly said: "sure, those factors you named are all plausible and in that sense correct but they're not the ones I named on my slides; therefore, the answer is incorrect". Admit it: if you grade your exams like that, you can just stop pretending and go ahead and use multiple-choice.
I studied at a small institution. At least I could go argue with my professor. I've heard some horror stories from friends who studied at Germany's large universities. There, short-answer and essay question exams are graded by an army of teaching assistants. However, these people quite literally don't give a sh*t, as they're paid poorly and the quality of their grading doesn't matter much to them. So, they go ahead and check if you answered exactly what the answer-guide says you should have answered. If you produced another answer that is well-argued and correct but chooses a different route of argumentation, well, tough luck.
Quite honestly, I believe that this can be much worse than a multiple-choice exam. You are still forced to try and read the professor's mind but you do not even get a hint what it might be that he wants you to put down. In a multiple-choice exam, at least you have options, so you have a general idea what kind of an answer the professor might expect.
Now, mind you, I do believe that with a well-worded open essay question you can get a much deeper look at a student's knowledge than you ever could with a multiple-choice exam. However, for that standard-issue, "I don't have much time for grading this", 200 student class-size undergrad exam, I do believe that it can be better to invest some time up-front into a well-formulated multiple-choice test that can be graded easily. It's better, in any case, than investing the same amount of time into sloppily grading a short-answer exam.
One last thing: you say that in your experience, short-answer exams and well-worded questions are correlated. I sure do believe that. However, I would like to offer an explanation: short-answer exams are much more work than multiple-choice exams. Therefore, only professors who are really serious about teaching are going to use them. These professors -- who are serious about their job as a teacher -- are very likely to produce well-worded questions in any kind of exam because, well, they care.
This is not a true and false issue. MC exams are valid for certain topics under certain conditions.
Many of your complaints only apply to poorly written MC questions.
Rob,
I give MC questions in my test, though never would an entire test be MC. The reason: easier to grade, sort out excellent from good.
For example, 15% on MC, 15% on match the columns. +3% for correct answer -1% for wrong answer.
MC, in my opinion cannot distinguish average students from good ones. Its also possible, as you said, that poor students might get a better grade just by guessing.
I agree with the limitations that multiple choice exams have but they can also be difficult and a good test of understanding if done well. Incorrect options can be offered that seem possible but show a gap in one particular area of understanding.
However, I agree with AgathaX's comment when she says that your real case is against bad teachers. Again, it is difficult to disagree. Unfortunately you find them everywhere and they are very difficult to get rid of. Unless they change the law they are something we just have to live with.