Thursday, April 28, 2011

Evidence from Medical Education that Quizzes Do Slow the Forgetting Curve

I read a recent post on the Questionmark Blog by John Kleeman, the founder of Questionmark, that I wanted to share with you. Often times, testing is discussed in a negative manner, but I enjoy finding research that commends the testing industry. There are many positive reasons for testing and slowing the forgetting curve is a great example.

Here is the post by Mr. Kleeman:
I’ve recently come across a really exciting set of results from a study of doctors-in-training that show quizzes really do slow down the forgetting curve.

To remind you, once we learn something, we start to follow a downwards “forgetting curve”. Psychology research says that taking quizzes and tests can reduce the forgetting curve by giving us retrieval practice, which helps retain information. But it’s rare to see this shown in action quite so vividly as in an experiment conducted with doctors-in-training and published in the respected Medical Education journal. (The citation is: Larsen, D. P., Butler, A. C. and Roediger III, H. L. (2009), Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Medical Education, 43: 1174–1181.)

In this experiment, Dr Douglas Larsen and colleagues Butler and Roediger from Washington University in St Louis divided some doctors-in- training into two groups of about 20 people each. The doctors were learning about two medical topics : status epilepticus and myasthenia gravis, abbreviated to SE and MG below.

Group A did the following:
  • Study session on both SE and MG
  • Quiz on SE immediately after the session and equal re-study time spent on MG
  • 2 weeks after study session, quiz on SE and equal re-study time on MG
  • 4 weeks after study session, quiz on SE and equal re-study time on MG
  • 6 months later, test on both SE and MG
Group B did the same with SE and MG switched round, ie
  • Study session on both SE and MG
  • Quiz on MG immediately after the session and equal re-study time spent on SE
  • 2 weeks after study session, quiz on MG and equal re-study time on SE
  • 4 weeks after study session, quiz on MG and equal re-study time on SE
  • 6 months later, test on both SE and MG
If spending time on re-study and taking a quiz have equal benefit to retention, then you would expect that both groups would perform about the same on the final test.

But if taking a quiz does actually aid retention and slow down forgetting, then you would expect that Group A would do better on SE than group B, and that group B would do better on MG than group A.

So what did happen? Here are the results for topic SE; you can see that group A initially scored an average of 78% and then knowledge reduced over time to 42% on the final test. However group B got a final score of 31%, much lower.

And here are the results for topic MG; group B, who did the earlier quizzes/tests on MG, scored an average of 36% on the final test, but group A, who’d just done the studying, got 19%.

These results show very clearly that taking the quizzes/tests helped people retain information vs. spending the same amount of time re-studying. And the study gives a dramatic picture of what happens in real learning – how people do forget, but also how quizzes and tests can reduce this.
Please visit the Questionmark Blog to find additional information on assessments and the testing industry. 

Wednesday, April 27, 2011

Storify: Make Stories Using Social Media

Storify is a really cool site to create stories using social media.  You simply search the web (Twitter, Flickr, Facebook, YouTube, etc.) for your story elements and then drag and drop them into the story line.  You can re-order the elements and also add text to give context to your readers. Then you can publish or embed the story.

Why should I make stories with Storify?
According to their site, "Millions of people are sharing content through social media. But these streams of information are quickly lost in the never-ending stream of updates. With Storify, you can put together the best Tweets, photos and videos to make stories that will be remembered."

Educational Applications?
  • My first thought is this format could used as a way to create skills standards for various occupations.
  • What a great way to create stories in journalism class following current events, natural disasters, etc.
  • Government classes could follow legislative events to create a storyline and study the legislative process.

Let us know how you would use Storify!

Thursday, April 21, 2011

Tired of Emailing Files and Photos to Yourself?...Try Dropbox!

I REALLY like Dropbox!

That pretty much sums it up. Your "Dropbox" is a special folder on your computer that allows you to simply drop in files and then they are instantly available on any of your other computers (desktop, laptop, iPad, iPhone, and/or other mobile devices).

The whole idea apparently is that the Dropbox folder is synched across the web and across platforms (Windows, Mac, Linux, iPad, iPhone, Android and BlackBerry). You can actually install Dropbox on other computers and have a local Dropbox folder on them, and all of these can synch up.

Even better, you can choose to share files with others (co-workers, clients, family and friends, etc.), as each file that you drag into the Public folder will get its own unique URL. On top of that, you can create file folders that you share just with other specific Dropbox users. The possibilities for this easy, inexpensive file sharing and synchronizing tool seem tremendous!

I also want you to know that Dropbox works well when you have large files that are simply too big to email.

It only takes a few minutes to download and did I mention that you get 2GB of storage for free?

Try Dropbox and let us know what you think!

Tuesday, April 19, 2011

How Do We Learn Best?...From Our Mistakes

I was talking with a few people this evening about work, education, families....basically life in general and now I'm sitting here watching a Texas Rangers vs. LA Angels of Anaheim baseball game (by the way, another nice win by the Rangers), but I keep reflecting on the evening's earlier conversations. Most of the stories that were told were hilarious mistakes of our youth and "other" people's misfortunes at work. I know that none of us would admit publicly to what we do wrong at work. I mean we do like to keep our jobs and our income, but most of us can talk about the mistakes of "others."

Again, all of these funny stories involved mistakes and it's amazing how we remember so many details about our own mistakes. So what is a "mistake" anyway?

According to, a "mistake" is defined as:
An error in action, calculation, opinion, or judgment caused by poor reasoning, carelessness, insufficient knowledge, etc.
Guess what? The definition never states that any of us are stupid for making mistakes in life or in school. Mistakes are often what we learn best from and we should expect mistakes from our children and our students.

We need to create an environment for students where they have the freedom to make mistakes. For them to understand that this is just "life" and mistakes are what drives us to succeed. After all Thomas Edison and the Wright Brothers had a plethora of mistakes, but they never lost their drive to create and to succeed.

Mistakes are just a part of learning (an important part in my opinion).

I know you are wondering why a testing blog is rambling on about creating an educational environment that condones making mistakes. After all, we want our students to have perfect scores on every test don't we? In a perfect world, yes would be the answer, but most of us aren't perfect and mistakes happen. There are also many reasons as to why a student may or may not score well on any given assessment.

From a psychometric point of view, the scores from a standardized test should create a normal (bell-shaped) curve, but this rarely happens. As I have said before on this blog, a test is "one" point-in-time piece of useful information. The following quote will provide you a good description of what constitutes a test?
A standardized test is a task or set of tasks given under standardized conditions and designed to assess some aspect of a person's knowledge, skill, or personality. A test provides a scale of measurement for consistent individual differences regarding some psychological concept and serves to line up people according to that concept. Tests can be thought of as yardsticks, but they are less efficient and reliable than yardsticks, just as the concept of verbal reasoning ability is more complex and less well understood than the concept of length. A test yields one or more objectively obtained quantitative scores, so that, as nearly as possible, each person is assessed in the same way. The intent is to provide a fair and equitable comparison among test takers. Green (1981, p. 1001) as cited in Jerome Sattler's Assessment of Children, 1992.
As Green states, tests are less reliable that the measurement of length and we therefore need to use caution and consider all factors prior to placing a  formal or informal "label" on a child. What educators need to do more of is to use the data that we collect to help students by building upon their strengths and remediating their weaknesses. I think we all too often throw the scores of most tests into a student's file without taking the time to evaluate strengths and weaknesses.

So as we head into the last few weeks of school and all of the testing commences, I hope you will look at each student's end-of-year assessments and do what is best for each individual student. Please consider all environmental or socio-economic factors and remember that any decision about students should include norm-referenced tests, informal assessments, observations, and interviews.

A test means so much more than just a score.

Thursday, April 14, 2011

Tag Galaxy: Visual Word Mapping

Tag Galaxy featuring "Claude Monet"
 Tag Galaxy is a cool mashup that uses Flickr images and a Flash interface. This tool can help students explore relationships between words and ideas as well as view pictures related to that word.

All you have to do is type in a word to search and then you are transferred to a galaxy with the original word at the center and the associated ideas orbiting around it. When you click on the word in the center, a globe populates with pictures from Flickr that are tagged with that word. Click on one of the orbiting words and tags related to that word start to orbit. Tag Galaxy is a great way to view information and visualize relationships.

A word of a little careful prior to sharing with students. Although I haven't seen any questionable pictures in the "globes" I have created, people upload all kinds of images and there could always be one that might cause some concern. I hope you enjoy Tag Galaxy
A "World" featuring Claude Monet.

Tuesday, April 12, 2011

Law, Public Safety, Security & Corrections Core Assessment

This NEW online assessment, from the CareerTech Testing Center, is an end-of-program assessment for students in law enforcement, criminal justice, security and or corrections programs. The assessment provides an indication of student mastery of basic knowledge and concepts necessary for success in careers in these areas.

The Law, Public Safety, Security, & Corrections Core Assessment was developed in partnership with the National Partnership for Careers in Law, Public Safety, Corrections and Security to provide a method to identify, measure, and ultimately recognize student achievement in law, public safety, security, and corrections programs.

The Knowledge and Skills Statements for the Law, Public Safety, Security and Corrections Cluster developed by the States’ Career Cluster Initiative are the heart of the certificate program. These standards identify the basic knowledge and skills that all entry-level workers in this sector need to be successful, regardless of the pathway or occupation they select.

The National Law, Public Safety, Security and Corrections Core Assessment is a 100-question assessment that measures student mastery of the basic knowledge and skills needed by all workers in the law and public safety sector. Students are provided meaningful feedback on their performance in ten key areas.

Students scoring 70% or higher on the National Law, Public Safety, Security and Corrections Core Assessment receive a certificate recognizing their accomplishment.

All assessment items were written by subject matter experts from several states and students from across the United States assisted with beta-testing. Information about the assessment, including ordering information ($12.00 per student per administration) can be obtained at

Please contact the CareerTech Testing Center at 800.522.5810 ext. 403 to order this new assessment today!

Friday, April 8, 2011

The Evolution of the Khan Academy

Sal Khan and the Khan Academy continue to evolve and progress over time. I've enjoyed and posted about their instructional videos a few times in the past, but they are expanding their services as he has gathered a greater following and additional funding. No longer a one man show, Sal and his team have added methods to track what your students are learning, adaptive assessment exercises, statistical analysis on student progress, a knowledge map for exercise progress, and classroom data for teachers and coaches. If all of that wasn't cool enough, students can now earn "badges" and "points" for learning. The more you learn, the more you earn. Hmmmm... isn't that the way life typically works anyway?

Watch the following Ted Talk featuring Salman Khan and Bill Gates then read below to learn more about their newest services:

How it works for students
Students can make use of our extensive video library, practice exercises, and assessments from any computer with access to the web.

Coaches, parents, and teachers
Coaches, parents, and teachers have unprecedented visibility into what their students are learning and doing on the Khan Academy.

Over 2100 videos
Our library of videos covers K-12 math, science topics such as biology, chemistry, and physics, and even reaches into the humanities with playlists on finance and history. Each video is a digestible chunk, approximately 10 minutes long, and especially purposed for viewing on the computer.

A world of exercises, with help along the way
Practice math at your own pace with our adaptive assessment exercises. You can start at 1+1 and work your way into calculus or jump right into whatever topic needs some brushing up.

Each problem is randomly generated, so you never run out of practice material. If you need a hint, every single problem can be broken down, step-by-step, with one click. If you need more help, you can always watch a related video.

Your stats, instantly
Every time you work on a problem or watch a video, the Khan Academy remembers what you've learned and where you're spending your time. We keep all of this data private but expose powerful statistics to each user and their coaches. You get at-a-glance information about everything you've been learning and whether or not you've been hitting your goals.

You can drill all the way down from a bird's-eye view of your profile into each and every exercise problem that you've ever worked on. You'll see real, hard data about your increasing mastery of math.

A map of knowledge
The knowledge map shows all of our exercise concepts. You can zoom in and out and pan around all the different exercises, just like on a normal map. You can start working anywhere on the map, and the Khan Academy will suggest the best exercises for you to work on. We'll even remind you when you need a review.

As you zoom out of the knowledge map, you'll find our challenges, which combine the concepts from multiple smaller exercises. Zooming all the way out of the map and completing all of our challenges is a sign of math mastery. We're constantly adding new exercises to be discovered.

Your classroom data...finally
Teachers and coaches can access all of their students' data. You can get a summary of class performance as a whole or dive into a particular student's profile to figure out exactly which topics are problematic. The class profile lets coaches glance at their dashboard and quickly figure out how to best spend their time teaching.

We've put a lot of energy into making sure that the Khan Academy empowers teachers by giving them access to the data they should've had for years. You'll know instantly if a student is struggling in multiplying fractions...or if she hit a streak and is now far ahead of the class.

Badges worth bragging about
We're full of game mechanics. As soon as you login, you'll start earning badges and points for learning. The more you challenge yourself, the more bragging rights you'll get.

We've heard of students spending hour after hour watching physics videos and 5th graders relentlessly tackling college-level math to earn Khan Academy badges. Some of the smaller badges are very easy, but the most legendary badges might require years of work.

Click HERE to visit the Khan Academy.

Wednesday, April 6, 2011

Twurdy - Search the Internet by Reading Level

Twurdy is a free search engine (powered by Google) that indicates the reading level of each page in a search result. Pages are given a reading level score, and results are color coded to make it easy for a user to recognize which results are likely to be most helpful.

Twurdy's rationale behind it's development is that...
Everyone has different reading abilities. Some people searching the web are university professors and others are 5 year old children. Twurdy has been created to provide people with access to search results that suit their own readability level.
 Twurdy's goal?
Twurdy's goal is to provide web searchers with information that is most appropriate for them. This will mean that 10 year olds doing school assignments don't have to click through difficult material to find something they can use. It will also mean that phd students do not have to click through websites designed for kids in order to find what they are looking for.
 How does Twurdy calculate the page readability level?
Twurdy uses custom designed readability software that includes information about the number of words on the page, the average number of syllables in each word, the average sentence length and more to determine a pages readability level.
What is the difference between the different search types?
  1. Just Twurdy - searches using Twurdy's basic algorithm with medium speed and medium results.
  2. Simple Twurdy - searches using Twurdy's simple algorithm for fast speed but less accurate results.
  3. Twurdy with Pop - searches using Twurdy's most complex algorithm which includes looking up the popularity of words within the text. It has a slower speed with more accurate results.
Here is an example of how Twurdy demonstrates a search for "St. Lucia Geography." (One of my favorite vacation spots!)

Monday, April 4, 2011

The Complete Educator’s Guide to Using Skype Effectively in the Classroom

I started to write a new post last week about Skype and I'm glad that I didn't because I checked my Twitter feed this morning and found a new blog post by Sue Waters on The Edublogger that was what I was wanting to post and more.

If you don't currently use Skype, it is a free application that allows you to call people from all over the world using the Internet. When you contact another person that uses Skype you talk or chat for free. Best of all you can tell when another user is online and what their status is so you know if they are available.

Click HERE to read "The Complete Educator’s Guide to Using Skype Effectively in the Classroom" where you will find every thing you need to know about Skype from…
  1. Setting up Skype
  2. Using Skype
  3. To using Skype effectively within your classroom
Whether you use interactive white boards, a data projector, web cams or just chat by IM, I find Skype to be an excellent cost-savings tool for the classroom or the office. I also think it creates many opportunities that you might not have otherwise. For example, you can now have in-class presentations from speakers that don't have to leave their office or from speakers or other classrooms from around the world.

If you use Skype or begin using Skype, I hope you will share your successes with us!

UPDATE: Skype in the Classroom.... A Dedicated Network for Teachers
Skype in the Classroom is a free community to help teachers everywhere use Skype to help their students learn. It’s a place for teachers to connect with each other, find partner classes and share inspiration. This is a global initiative that was created in response to the growing number of teachers using Skype in their classrooms.
Related Posts Plugin for WordPress, Blogger...