Evidence from Medical Education that Quizzes Do Slow the Forgetting Curve

I read a recent post on the Questionmark Blog by John Kleeman, the founder of Questionmark, that I wanted to share with you. Often times, testing is discussed in a negative manner, but I enjoy finding research that commends the testing industry. There are many positive reasons for testing and slowing the forgetting curve is a great example.

Here is the post by Mr. Kleeman:

I’ve recently come across a really exciting set of results from a study of doctors-in-training that show quizzes really do slow down the forgetting curve.

To remind you, once we learn something, we start to follow a downwards “forgetting curve”. Psychology research says that taking quizzes and tests can reduce the forgetting curve by giving us retrieval practice, which helps retain information. But it’s rare to see this shown in action quite so vividly as in an experiment conducted with doctors-in-training and published in the respected Medical Education journal. (The citation is: Larsen, D. P., Butler, A. C. and Roediger III, H. L. (2009), Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Medical Education, 43: 1174–1181.)

In this experiment, Dr Douglas Larsen and colleagues Butler and Roediger from Washington University in St Louis divided some doctors-in- training into two groups of about 20 people each. The doctors were learning about two medical topics : status epilepticus and myasthenia gravis, abbreviated to SE and MG below.

Group A did the following:

• Study session on both SE and MG
• Quiz on SE immediately after the session and equal re-study time spent on MG
• 2 weeks after study session, quiz on SE and equal re-study time on MG
• 4 weeks after study session, quiz on SE and equal re-study time on MG
• 6 months later, test on both SE and MG

Group B did the same with SE and MG switched round, ie

• Study session on both SE and MG
• Quiz on MG immediately after the session and equal re-study time spent on SE
• 2 weeks after study session, quiz on MG and equal re-study time on SE
• 4 weeks after study session, quiz on MG and equal re-study time on SE
• 6 months later, test on both SE and MG

If spending time on re-study and taking a quiz have equal benefit to retention, then you would expect that both groups would perform about the same on the final test.

But if taking a quiz does actually aid retention and slow down forgetting, then you would expect that Group A would do better on SE than group B, and that group B would do better on MG than group A.

So what did happen? Here are the results for topic SE; you can see that group A initially scored an average of 78% and then knowledge reduced over time to 42% on the final test. However group B got a final score of 31%, much lower.

And here are the results for topic MG; group B, who did the earlier quizzes/tests on MG, scored an average of 36% on the final test, but group A, who’d just done the studying, got 19%.

These results show very clearly that taking the quizzes/tests helped people retain information vs. spending the same amount of time re-studying. And the study gives a dramatic picture of what happens in real learning – how people do forget, but also how quizzes and tests can reduce this.

Please visit the Questionmark Blog to find additional information on assessments and the testing industry. 

Storify: Make Stories Using Social Media

Storify is a really cool site to create stories using social media.  You simply search the web (Twitter, Flickr, Facebook, YouTube, etc.) for your story elements and then drag and drop them into the story line.  You can re-order the elements and also add text to give context to your readers. Then you can publish or embed the story.

Why should I make stories with Storify?
According to their site, "Millions of people are sharing content through social media. But these streams of information are quickly lost in the never-ending stream of updates. With Storify, you can put together the best Tweets, photos and videos to make stories that will be remembered."

Educational Applications?

• My first thought is this format could used as a way to create skills standards for various occupations.
• What a great way to create stories in journalism class following current events, natural disasters, etc.
• Government classes could follow legislative events to create a storyline and study the legislative process.

Let us know how you would use Storify!

Tired of Emailing Files and Photos to Yourself?...Try Dropbox!

I REALLY like Dropbox!

That pretty much sums it up. Your "Dropbox" is a special folder on your computer that allows you to simply drop in files and then they are instantly available on any of your other computers (desktop, laptop, iPad, iPhone, and/or other mobile devices).

The whole idea apparently is that the Dropbox folder is synched across the web and across platforms (Windows, Mac, Linux, iPad, iPhone, Android and BlackBerry). You can actually install Dropbox on other computers and have a local Dropbox folder on them, and all of these can synch up.

Even better, you can choose to share files with others (co-workers, clients, family and friends, etc.), as each file that you drag into the Public folder will get its own unique URL. On top of that, you can create file folders that you share just with other specific Dropbox users. The possibilities for this easy, inexpensive file sharing and synchronizing tool seem tremendous!

I also want you to know that Dropbox works well when you have large files that are simply too big to email.

It only takes a few minutes to download and did I mention that you get 2GB of storage for free?

Try Dropbox and let us know what you think!

How Do We Learn Best?...From Our Mistakes

I was talking with a few people this evening about work, education, families....basically life in general and now I'm sitting here watching a Texas Rangers vs. LA Angels of Anaheim baseball game (by the way, another nice win by the Rangers), but I keep reflecting on the evening's earlier conversations. Most of the stories that were told were hilarious mistakes of our youth and "other" people's misfortunes at work. I know that none of us would admit publicly to what we do wrong at work. I mean we do like to keep our jobs and our income, but most of us can talk about the mistakes of "others."

Again, all of these funny stories involved mistakes and it's amazing how we remember so many details about our own mistakes. So what is a "mistake" anyway?

According to Dictionary.com, a "mistake" is defined as:

An error in action, calculation, opinion, or judgment caused by poor reasoning, carelessness, insufficient knowledge, etc.

Guess what? The definition never states that any of us are stupid for making mistakes in life or in school. Mistakes are often what we learn best from and we should expect mistakes from our children and our students.

We need to create an environment for students where they have the freedom to make mistakes. For them to understand that this is just "life" and mistakes are what drives us to succeed. After all Thomas Edison and the Wright Brothers had a plethora of mistakes, but they never lost their drive to create and to succeed.

Mistakes are just a part of learning (an important part in my opinion).

I know you are wondering why a testing blog is rambling on about creating an educational environment that condones making mistakes. After all, we want our students to have perfect scores on every test don't we? In a perfect world, yes would be the answer, but most of us aren't perfect and mistakes happen. There are also many reasons as to why a student may or may not score well on any given assessment.

From a psychometric point of view, the scores from a standardized test should create a normal (bell-shaped) curve, but this rarely happens. As I have said before on this blog, a test is "one" point-in-time piece of useful information. The following quote will provide you a good description of what constitutes a test?

A standardized test is a task or set of tasks given under standardized conditions and designed to assess some aspect of a person's knowledge, skill, or personality. A test provides a scale of measurement for consistent individual differences regarding some psychological concept and serves to line up people according to that concept. Tests can be thought of as yardsticks, but they are less efficient and reliable than yardsticks, just as the concept of verbal reasoning ability is more complex and less well understood than the concept of length. A test yields one or more objectively obtained quantitative scores, so that, as nearly as possible, each person is assessed in the same way. The intent is to provide a fair and equitable comparison among test takers. Green (1981, p. 1001) as cited in Jerome Sattler's Assessment of Children, 1992.

As Green states, tests are less reliable that the measurement of length and we therefore need to use caution and consider all factors prior to placing a  formal or informal "label" on a child. What educators need to do more of is to use the data that we collect to help students by building upon their strengths and remediating their weaknesses. I think we all too often throw the scores of most tests into a student's file without taking the time to evaluate strengths and weaknesses.

So as we head into the last few weeks of school and all of the testing commences, I hope you will look at each student's end-of-year assessments and do what is best for each individual student. Please consider all environmental or socio-economic factors and remember that any decision about students should include norm-referenced tests, informal assessments, observations, and interviews.

A test means so much more than just a score.

Tag Galaxy: Visual Word Mapping



Tag Galaxy featuring "Claude Monet"

 Tag Galaxy is a cool mashup that uses Flickr images and a Flash interface. This tool can help students explore relationships between words and ideas as well as view pictures related to that word.

All you have to do is type in a word to search and then you are transferred to a galaxy with the original word at the center and the associated ideas orbiting around it. When you click on the word in the center, a globe populates with pictures from Flickr that are tagged with that word. Click on one of the orbiting words and tags related to that word start to orbit. Tag Galaxy is a great way to view information and visualize relationships.

A word of caution....be a little careful prior to sharing with students. Although I haven't seen any questionable pictures in the "globes" I have created, people upload all kinds of images and there could always be one that might cause some concern. I hope you enjoy Tag Galaxy! 

A "World" featuring Claude Monet.