I want to thank
John Kleeman, the founder of
Questionmark, for reminding us of the importance of assessment feedback and its importance in learning. His
blog post below links to several reports that support his thoughts and I agree with their findings, but I would like to add a few thoughts of my own before you read the post in its entirety.
First, I hope that any learner will want to improve their score by use of feedback, but I'm not always sure that every learner will accomplish this on their own (for many reasons). As John points out:
"You must get the learner to evaluate the feedback and adjust their thinking as a result of the feedback, i.e. process the feedback and do something with it."
It can be a difficult thing to get an individual to change their thought process, even with feedback that demonstrates the obvious, so I believe that it's important for anyone that administers a test, whenever possible, to understand and consider all factors that affect an individual's test score (environmental, socio-economic, education, training, etc.) and to keep these factors in mind when interpreting results and developing effective remediation plans.
I also think it's important to note feedback is needed for more than just the student and that tests scores should be aggregated for group analysis as they provide feedback to instructors seeking to improve the effectiveness of their curriculum and instructional methods. Depending upon your organization, the aggregated results can also be used for overall program (organizational/statewide) improvement. I really wish that more people would understand that a test is a very effective educational tool that reinforces learning at all levels.
Here is
John's post and some of it is Questionmark software specific, but it provides another great reference point on the use of tests for learning:
Effective Assessment Feedback Needs Motive, Opportunity and Means
Assessment feedback, whether it relates to a question, a topic or an entire assessment, has tremendous value – but only if learners make practical use of it! I’d like to share some solid approaches about feedback and talk about how a couple of Questionmark reports can help you put them into practice.
From Andrew Morgan (quoted in Valerie Shute’s excellent ETS research report on feedback), we get the concept that to be effective and useful, feedback needs the following:
- Motive – the learner wants to follow the feedback
- Opportunity – the learner has it in time to use it: it’s not given too late for action
- Means – the learner is able to use it: the feedback is actionable
Another good way to think about feedback comes from Dr Steve Draper of the University of Glasgow School of Psychology in his presentation at eAssessment Scotland in 2012:
“There is no point in giving feedback to a learner unless the learner acts on it: does something concrete and differently because of it”.
Feedback that the learner doesn’t read isn’t valuable.
Feedback that the learner reads but doesn’t process isn’t valuable.
You must get the learner to evaluate the feedback and adjust their thinking as a result of the feedback, i.e. process the feedback and do something with it.
I’ve been wondering about how you can apply these concepts using the Questionmark coaching report when presenting an assessment score as feedback.
Most learners are motivated to use their score achieved in a test as feedback; they want to get a higher score next time. And if they can take a test again, they have the opportunity to use the feedback. But a score on its own is just a number. How can you help learners use their scores as catalysts for action?
Clearly, a score is more valuable if it can be compared to something, and there are three obvious comparisons:
- Ipsative, comparing score to previous attempts: have you done better than last time?
- Criterion referenced, comparing score to a benchmark: have you reached the desired standard
- Normative, comparing score to how others do: how do you compare to your peers?Questionmark’s Transcript report lets learners view all their results and see how they improve or change over time.
Questionmark’s Transcript report lets learners view all their results and see how they improve or change over time. And Questionmark’s Coaching report includes the concept of benchmarks – you can set a benchmark for the assessment and for each topic. What you may not know is that the Coaching report allows you to compare a score against the average of a group of other test-takers. You can define the group of people by a variety of demographics and then display how the participant compares against their scores.
Giving learners information about how they compare to others can be a powerful motivator; I encourage you to explore this capability of the Questionmark Coaching report.
For more on general concepts of processing feedback, see Steve Draper’s interesting page here. Questionmark users can see more information on the Coaching report comparison options on the Questionmark support site. And if you want to hear more from me about assessment feedback, I’ll be speaking about it at the Questionmark user conference in March.
MDZE3SGDZH9Y