In this post, I explore why multiple choice quizzes are desirable in remote learning, the research around how to do them well and how to use Google Forms to get the most desirable outcomes.
Remote learning’s feedback problem
One of the challenges facing teachers in remote learning is knowing that students are getting less feedback that they would otherwise get in a physical classroom. We usually monitor students as they work independently in order to pick up any glaring misconceptions early. As Rosenshine suggests, independent practice by students should be done with a high success rate. If not, they are essentially embedding misconceptions that are difficult to unpick later on.
During remote learning when students are completing independent work unsupervised, a solution might be to do more video-conferencing with students. There are fantastic ways to check for understanding and provide instant feedback in a video conference during the guided practice phase of learning, but there are also problems with thinking you can simply extend the amount of time spent in video conferences: it eats into the time teachers need to prepare resources, it exacerbates “Zoom fatigue” for everyone and it disadvantages students with poor internet connections.
Another solution is written feedback, but this also has its problems: written feedback may not actually be read, or received too late to have much of an impact. Plus it takes a long time! Compared to a teacher monitoring student work and quickly providing feedback while the student is practising, written feedback is often not effective.
Digital solutions can provide instant feedback to students
I’ve previously suggested ways to optimise the quality of independent practice in remote learning: We can provide answer sheets and knowledge organisers for self-quizzing, and I recently blogged about using Quizlet as an activity that instantly gives students feedback about their responses. Because time is a precious commodity for teachers—doubly so during remote learning—digital resources that provide students with instant feedback should be fully explored amongst the strategies used to support successful independent learning. Recently I’ve explored creating digital multiple choice quizzes with much success: upon completion, students immediately receive information about how well they did and my year level team gets instant information about what re-teaching we need to prioritise. Not only that, students actually enjoy doing them!
So how do you write a good multiple choice quiz?
1: USE THE QUIZ FOR REVIEW
The benefit of a quiz does not only lie in the assessment information it yields. The quiz itself enhances learning by prompting students to bring to mind prior learning, strengthening the student’s understanding and embedding knowledge more securely in long-term memory. This is commonly known as the ‘testing effect.’ Therefore, quiz content should target previously studied material. Blake Harvard recommends a ‘Last Lesson, Last Week, Last Month‘ structure for review which I’ve found very useful.
2: INTERLEAVE QUESTIONS
While studying similar problems together (blocked practice) is useful for novices to gain fluency in a skill, mixing different problem types (interleaved practice) leads to better long-term learning. Therefore review quizzes should include different problem types arranged in a more-or-less unpredictable order. I’ve found that following the Last Lesson, Last Week, Last Month review structure leads to natural interleaving.
3: DEVELOP QUESTIONS THAT REQUIRE HIGHER ORDER THINKING
Daisy Christodoulou argues convincingly that multiple choice questions can prompt higher order thinking. She offers the following example:
How did the Soviet totalitarian system under Stalin differ from that of Hitler and Mussolini?
A. It built up armed forces.
B. It took away human rights.
C. It made trade unions illegal.
D. It abolished private land ownership.
So MCQs can prompt higher order thinking, but it’s true that many MCQs only require fact recall. While I don’t think there’s anything wrong with assessing factual knowledge, it’s good practice to think about Bloom’s taxonomy when writing questions and prompt a variety of different types of thinking. What does this mean for me, teaching Year 3 maths? Here are some different question types I’ve written:
Multiple representation questions require students to understand how concepts can be represented in different ways.
Example: What is the time on this clock?
(B) Half past two
(C) 6 o’clock
(D) Half past three
Process questions ask students to evaluate the processes used to solve a given question.
Example: Which of these are strategies for finding 6 x 5?
(A) Halve 6, then multiply by 10
(B) 5 + 5 + 5 + 5 + 5 + 5
(C) Halve 5, then multiply by 10
(D) 6 + 6 + 6 + 6
(E) 5 + 5 + 5 + 5 + 5
True and false questions ask students to evaluate the veracity of given statements.
Example: Which statements are true?
(A) August is the month after July.
(B) The autumn months are March, April and May.
(C) September is the month after October.
(D) The summer months are January, February and March.
(E) 30 days have September, April, June and December.
Reasonable estimate questions ask students to retrieve information to evaluate the reasonableness of given estimates. The example below requires students to recall the average weight of a newborn baby in order to determine which given estimates are reasonable.
Example: What are the most reasonable estimates for the mass of this baby?
(A) 5 kg
(B) 6 kg
(C) 50 kg
(D) 60 kg
(E) 1 kg
4: CREATE DISTRACTORS THAT TARGET MISCONCEPTIONS
A good multiple choice question depends on the quality of the “distractor” choices: They need to be “plausible, but not so plausible that they are unfair” (Bjork, 2012). If the distractors are implausible, students can use a process of elimination to find the answer, but their success can’t be used to indicate secure knowledge. But if the distractors target common misconceptions, then student responses can provide teachers with useful information and can also prompt useful class discussion.
5: INCLUDE MORE THAN ONE CORRECT ANSWER
Including more than one answer reduces the possibility for students to guess the correct answer and also inherently makes the question more rigorous. It is often the case that students decide that one question is correct, then neglect to carefully read the remaining choices. If you include, say, two answers, students have to read and evaluate each option carefully. I’ve set a constraint on myself to always write multiple choice questions with 5 options, two of which are correct, and I’ve found it pushes me to write more interesting questions.
How do you use Google Forms properly?
1: SET UP A QUIZ
Go to Google Forms and create a new form. Give it a title and include questions asking for the student’s name and class. Go to Settings > Quizzes and select Make this a quiz. This allows you to assign point values to each question and enable auto-marking.
Note: If you want to include a question that requires uploading a file, Google requires you to create the form in an unshared drive. In this case, I recommend creating a folder for the quiz and sharing the folder with your colleagues so they can access the student data.
2. CREATE A TEMPLATE QUESTION AND DUPLICATE IT
If you want to include two correct options for each question as I’ve suggested, you’ll want to choose Checkboxes rather than Multiple choice (which only allows one response). Create five options. Check Required so students can’t submit without attempting the question. Click on the three dots and select Response validation to require students to select exactly 2 options. Click on Answer key to set the number of points per question (I stick to 1 point per question) and check 2 options as correct. I also check Shuffle option order so I don’t have to think about it. Once you’ve got these settings correct, duplicate the question as many times as you like and then you can focus on writing your questions.
3. CREATE AN ASSIGNMENT IN GOOGLE CLASSROOM AND ATTACH THE QUIZ
Once you are happy with your quiz, create an assignment in Google Classroom (not a quiz assignment) and attach the quiz from your Google Drive.
4. RETURN THE QUIZ
Once your students have submitted their responses, they will automatically have access to their results. If you are using Google Classroom, go the assignment and click Import grades, then select students who have submitted their work and click Return to digitally return the assignment.
5. ANALYSE THE RESULTS
One of the neatest things about using Google Forms is that you automatically get data on trends within your cohort. It informs you of frequently missed questions and provides a breakdown of each question. Here’s an example:
It’s clear that a good chunk of students don’t have a solid understanding of a phrase such as “5 to 10” and that some reteaching is required here.
You can also view results in an automatically generated spreadsheet. To make sense of the data, I sort the students by score. This allows you to quickly identify which students struggled and make instructional decisions based on the data. Another useful tool is to use conditional formatting to highlight correct responses. This doesn’t take long and results in something like this:
This allows a quick visual analysis which enables you to prioritise what reteaching will benefit a larger percentage of students. In the image above, the presence of white near the top of the column and larger amounts of white cels, tells me that students struggled with interpreting the meaning of open number lines and this can be subsequently addressed through reteaching.
6. RESPOND TO THE RESULTS
Finally, it’s not enough to simply assign quizzes, they need to be used formatively. Here are a few things I do with the results from review quizzes:
- Engage students in reflective conversations. What questions did they answer confidently? What was challenging? Involve students in what you decide to reteach. Respond immediately with an explanation and guided practice.
- Plan reteaching around common misconceptions. Which questions were particularly challenging for the class as a whole? Don’t assume that reteaching will only benefit students who were unsuccessful with that question. It’s likely that some students who chose the correct options in the quiz also found that question challenging.
- Identify students for small group intervention. Who really struggled? Why did they struggle? I worked through an old review quiz with one of my students who received a low score, modelling strategies such as checking each option one-by-one. The next week saw a huge improvement in scores. My guess is that her barrier was only partially due to insecure understanding, but more to do with her unfamiliarity with the quiz format.
- An “Ah-Ha” Moment with Spaced Practice in the Classroom by Blake Harvard | link
- Closed questions and higher order thinking by Daisy Christodoulou | link
- Multiple choice questions, part two by Daisy Christodoulou | link
- Research on multiple choice questions by Daisy Christodoulou | link
- Diagnostic Questions: Is There Value in Just One? by Caroline Wylie and Dylan Wiliam | link
- When is assessment learning-oriented? by Dylan Wiliam | link