Cheating Your Way Through High School and College: The Numbers

Academic cheating seems to be growing at an ever alarming rate. Cheating occurs at every level of education and certification and it includes students, educators, and administrators. Technology has also given cheaters new methods for cheating and, unfortunately, you can find most of these methods demonstrated on YouTube (you can find additional resources on cheating HERE).

Best College Reviews created an interesting infographic about cheating in high school and college that I wanted to share with you. They provide resons for cheating and how cheating occurs among other interesting factoids.

Please take a look at their website to find additional resources on cheating (listed below the infographic).


Source: BestCollegeReviews.org (please follow the link for additional resources on cheating)

The Benefits of Educational Quizzes and Tests

John Kleeman shared on Questionmark's Blog the ten benefits of quizzes and tests in educational practice as proposed by psychology experts Henry L. Roediger III, Adam L. Putnam and Megan A. Smith in a recent paper, “Ten Benefits of Testing and Their Applications to Educational Practice”.

Here is John's summary and understanding of the paper:

1. Retrieval aids later retention. There is clear evidence from psychological experiments that practicing retrieval of something after learning it, for instance by taking a quiz or test, makes you more likely to retain it for the long term.
2. Testing identifies gaps in knowledge.
3. Testing causes students to learn more from the next study episode. Essentially it reduces forgetting which makes the next related study area more productive.
4. Testing produces better organization of knowledge by helping the brain organize material in clusters to allow better retrieval.
5. Testing improves transfer of knowledge to new contexts. There are several experiments referenced in the paper where tests and quizzes help transfer and application of knowledge.
6. Testing can facilitate retrieval of material that was not tested. Surprisingly there are circumstances where quizzes or tests, particularly if delayed, can help people retrieve/retain information that was related to that asked but not actually asked in the questions.
7. Testing improves metacognitive monitoring – by giving students scores or self-assessments, they can better predict their knowledge and be more confident about what they know and what they need to know.
8. Testing prevents interference from prior material when learning new material. If you have a test after learning one set of material before learning another set of material, it can make it less likely that the second session will
9. Testing provides feedback to instructors and lets them know what is learned or what is not.
10. Frequent testing encourages students to study. Having frequent quizzes and tests motivates study and reduces procrastination.

 You can see their paper “Ten Benefits of Testing and Their Applications to Educational Practice” in Psychology of Learning and Motivation, Vol 55. It’s also available for download on Professor Roediger’s publications page, in the list of papers from 2011, at http://psych.wustl.edu/memory/publications/.

Test Above Knowledge Level: Use Scenario Based Questions

I read a post on Questionmark's blog by John Kleeman that I wanted to share with you below. I think Kleeman does a great job of providing a simple example of how scenario based questions out perform knowledge or "definition" questions. This is a point that I continually try and stress to subject matter experts when they begin writing test questions. I believe the "definition" questions take less thought and effort to create and that is why subject matter experts may opt to write these type of questions, but I think it's also important to help them understand how scenario questions out perform knowledge questions upon statistical review. Other benefits of scenario questions are that they are better at assessing critical thinking skills and they add to the reading level of a test.

Thanks to John for providing us with another great post that I can share with you and with subject matter experts. Here is the post in its entirety:

Here’s the one piece of advice I’d give above all others to anyone creating quizzes, tests or exams: Test above knowledge.

You may be familiar with Bloom’s taxonomy of learning objectives, which is shown in the diagram below. At the lowest level is Knowledge; questions that test Knowledge ask for simple remembering of facts. At the highest level is Evaluation; questions that test Evaluation require participants to use judgement.

 

It’s great if you can write questions that assess at the highest levels of Bloom’s Taxonomy, but many organizations have a lot to gain by asking questions at any of the levels above Knowledge in the pyramid. Although there are times when testing facts can be useful, it’s usually better to test the application or comprehension of those facts.

In today’s world, where facts are easily googleable, it is the use and understanding of those facts in the real world that is genuinely useful. By testing above knowledge, you are checking not just that the participant knows something but that they can apply it in some scenario. This is more valid and more realistic — and for most applications it is also more useful.

Here is a simple example to illustrate the point:

 

What does a yellow traffic light mean?

• Stop
• Go
• Caution

This is purely a factual, knowledge question.

But here, the question requires that the respondent to apply to meaning of a yellow traffic light to an actual situation:

If you are driving toward an intersection and the light turns from yellow to red, what should you do?

• Speed up and cross the intersection
• Stop suddenly
• Stop gradually

This is a very simple example, but I hope it makes you realize that converting factual questions to scenarios is not very hard.

I’d encourage you to consider using scenarios in your questions: Ask people to apply their knowledge, not just prove that they know some facts. Have your test-takers apply what they know to actual situations.

March Madness: Let the Testing Begin

My NCAA basketball tournament brackets have once again become my folly after only the first weekend of the tournament so I began to think of what "March Madness" really means for me in education... the beginning of the testing season.

I think there are many valid and useful reasons for testing so the "Madness" I refer to only means how busy we get during this time of year. To help you with your testing "Madness," I have created a list of twenty resources that will help you understand the reasons for testing, the testing process, test interpretation, and remediation.

I hope the list below will help you gain a better understanding of why we test and how important the process and the results are for your students and your program.

The CareerTech Testing Matrix (The Competency Test Process)

Understanding the “Depth” of Testing

Effectively Communicating the Measurement of Constructs to Stakeholders

Testing Integrity: Issues and Recommendations for Best Practices

Create a successful testing environment

Follow the recommended time for administering tests

Establish rapport with examinees

Limit test anxiety

Provide test security and eliminate cheating

Understanding the numbers I, II, III, IV

Interpreting test scores

The Pygmalion Effect: Are You Guilty?

Analyze individual tests scores across all levels - instructor, district, and state

Provide positive reinforcement and build upon an examinee’s relative strengths

Remediate an examinee’s relative weaknesses

Implement improvements across all levels

Reporting Test Results to Parents

10 Questions Every Parent and Student Should Ask About Testing

Study Tips and Skills

Read and learn “The Secret of Writing Multiple Choice Test Items”

Ten Benefits of Quizzes and Tests in Educational Practice

John Kleeman

I found an interesting post by John Kleeman on the Questionmark Blog that I wanted to share with you. John is one of the few people that I've found that frequently provides information on the psychology of assessment and I'm thankful that he allows us to share his thoughts with you.

Here is John's post in its entirety:

I’d like to share ten benefits proposed by psychology experts Henry L. Roediger III, Adam L. Putnam and Megan A. Smith in a recent paper, “Ten Benefits of Testing and Their Applications to Educational Practice”.

Here is my summary and understanding:

1. Retrieval aids later retention. There is clear evidence from psychological experiments that practicing retrieval of something after learning it, for instance by taking a quiz or test, makes you more likely to retain it for the long term.

2. Testing identifies gaps in knowledge.

3. Testing causes students to learn more from the next study episode. Essentially it reduces forgetting which makes the next related study area more productive.

4. Testing produces better organization of knowledge by helping the brain organize material in clusters to allow better retrieval.

5. Testing improves transfer of knowledge to new contexts. There are several experiments referenced in the paper where tests and quizzes help transfer and application of knowledge.

6. Testing can facilitate retrieval of material that was not tested. Surprisingly there are circumstances where quizzes or tests, particularly if delayed, can help people retrieve/retain information that was related to that asked but not actually asked in the questions.

7. Testing improves metacognitive monitoring – by giving students scores or self-assessments, they can better predict their knowledge and be more confident about what they know and what they need to know.

8. Testing prevents interference from prior material when learning new material. If you have a test after learning one set of material before learning another set of material, it can make it less likely that the second session will.

9. Testing provides feedback to instructors and lets them know what is learned or what is not.

10. Frequent testing encourages students to study. Having frequent quizzes and tests motivates study and reduces procrastination.

You can see their paper “Ten Benefits of Testing and Their Applications to Educational Practice” in Psychology of Learning and Motivation, Vol 55. It’s also available for download on Professor Roediger’s publications page, in the list of papers from 2011, at http://psych.wustl.edu/memory/publications/.

Teacher of Thousands: Piyush Patel

Here is a TEDxOStateU video entitled "Teacher of Thousands: Piyush Patel."


Piyush Patel founded and runs a company that generates virtual training programs for creative professionals who produce films, video games, commercials and other projects. Located in Oklahoma City, PL Studios Inc.,  along with the Digital-Tutors division, works with clients including Disney, NASA, Nike, Dreamworks, Pixar, EA Sports and Mercedes-Benz.

Piyush Patel

In 2010, the company was recognized as an official honoree of the broadband category at the 14th annual Webby Awards and was listed as one of the 50 Most Usable Rich Internet Applications of the year by the 2009 O’Reilly’s Inside RIA.

Patel is an advisory board member for the Oklahoma Technology Council, a member of the AIGA Oklahoma board of directors, a member of the Oklahoma State Chamber board of directors and the former president for the Oklahoma Chapter of Entrepreneurs Organization. He was a tenured professor at Northern Oklahoma College before founding PL Studios, Inc.

Patel has also been instrumental in helping the CareerTech Testing Center and the Oklahoma Department of Career and Technology Education in the development of our skills standards (knowledge, skills, and abilities needed to perform related jobs within an industry) for Motion Graphics Artist, 3D Animator, and 3D Modeler. Digital-Tutors has provided their endorsement of these standards which serve as the foundation for competency-based instruction in these areas.

Congratulations to Piyush on a great TEDx talk and for having the idea to create an instructional concept that can change the lives of thousands from across the globe!

Howard Gardner on Multiple Intelligences and New Forms of Assessment

I wanted to share an interesting video by Harvard University professor Howard Gardner that I found on Edutopia:


Howard Gardner is the John H. and Elisabeth A. Hobbs Professor of Cognition and Education at the Harvard Graduate School of Education. He also holds positions as Adjunct Professor of Psychology at Harvard University and Senior Director of Harvard Project Zero. Among numerous honors, Gardner received a MacArthur Prize Fellowship in 1981. He has received honorary degrees from twenty-nine colleges and universities, including institutions in Bulgaria, Chile, Greece, Ireland, Israel, Italy, South Korea and Spain.

He is the author of twenty-eight books translated into thirty-two languages, and several hundred articles, Gardner is best known in educational circles for his theory of multiple intelligences, a critique of the notion that there exists but a single human intelligence that can be adequately assessed by standard psychometric instruments.

Also read: An Interview with the Father of Multiple Intelligences from Edutopia.

Free Gift: The Secret of Writing Multiple Choice Test Items

One of the things we have learned over the years at the CareerTech Testing Center is that Subject Matter Experts are definitely experts in their field, but many aren't aware of how to design an effective test question. Therefore, we created one of our most popular posts, "The Secret of Writing Multiple-Choice Test Items." (This post led to the creation of the FREE GUIDE that you can can access below.

We believe that these "secrets" can help any instructor or test developer, at ANY level of instruction, develop more effective test instruments. We hope you take a look and remember... It's FREE!
(Download free copy from Yudu link below)

Click to launch the full edition in a new window
Digital Publishing with YUDU

Assessment Types and their Uses: Formative

Julie Delazyn
Questionmark

Here is a the second in a series of posts from the Questionmark Blog written by Julie Delazyn, Public Relations/Marketing for Questionmark that I wanted to share with you:

Assessments have many different purposes, and to use them effectively it’s important to understand their context and uses within the learning process.

Last week I wrote about diagnostic assessments, and today I’ll explore formative assessments.

Typical uses:

• Strengthening memory recall and correcting misconceptions
• Promoting confidence in one’s knowledge
• Enhancing learning by directing attention to and creating intrigue about a given subject
• Measuring learners’ knowledge or skills and telling them how they’re doing
• Giving learners search and retrieval practice and prescriptive feedback

Types:

• Quizzes
• Practice tests and exams
• Self-assessments

Stakes: low

Example: An instructor gives a quiz to help reassure students that they’re actually learning — or alert them that they are not learning and provide feedback to correct any misconceptions. Students can use this feedback as a study guide to understand where they’re going right and where they’re going wrong. Students also benefit from the search and retrieval practice they’ve had while taking the quiz – which can help them remember the material in the future. Formative assessments give instructors a way to ask students: “Did you get that?” Sometimes, a series of quizzes is used to collect data that contribute to overall grades – but generally, formative assessments serve as check-ups on learners’ understanding and guideposts for further progress.

For a fuller analysis of assessments and their uses check out the white paper, Assessments Through the Learning Process. You can download it free here, after login. Another good source for testing and assessment terms is our glossary.

In the coming weeks I’ll take a look at three remaining assessment types:

• Needs
• Reaction
• Summative

Assessment Types and their Uses: Diagnostic

Here is a recent post from the Questionmark Blog written by Julie Delazyn, Public Relations/Marketing for Questionmark that I wanted to share with you:

Assessments have many different purposes, and to use them effectively it’s important to understand their context and uses within the learning process. I’ll explore each of these five key assessment types over the next few weeks:

• Diagnostic
• Formative
• Needs
• Reaction
• Summative

Let’s start with diagnostic assessments.

Typical uses:

• Identifying the needs and prior knowledge of participants for the purpose of directing them to the most appropriate learning experience
• Determining knowledge and identifying skills gaps and needs
• Placing learners in appropriate courses and tailor instruction to their needs
• Providing instructors and mentors information on a student’s abilities
• Giving feedback to participants and providing recommendations for products, services and/or learning activities
• Setting benchmarks for comparison with post-course tests
• Analyzing personality traits in order to predict behaviors
• Creating intrigue about the content of a learning activity, which can in turn actually enhance the learning experience

Types:

• Pre-tests
• Placement tests
• Self-diagnostic tools
• Personality assessments
• Stakes: low/medium

Example: A diagnostic assessment might report that a learner has mastered every competency in using Microsoft Word but can only perform 50 percent of those required to use Excel. The results of the assessment would prescribe a course on Excel. In addition, a diagnostic assessment can help place students within suitable learning experiences by asking questions such as, “Do you prefer instructor-led training or online training?”

For a fuller analysis of assessments and their uses check out the white paper, Assessments Through the Learning Process. You can download it free here, after login. Another good source for testing and assessment terms is our glossary.

 

What is Ipsative Assessment and Why Would I Use It?

I read a recent post on the Questionmark Blog by John Kleeman, the founder of Questionmark, that I wanted to share with you. I have long been a fan of ipsative assessment since it allows you to compare a test-takers results against their own previous results. This is a great method for assessment when you are dealing with individual students, students in special education, those involved in career and technical education, and employees. The basic question you are trying to answer is, "Are their skills better today than they were yesterday?"

Please take a few minutes to read the following post by Mr. Kleeman and think about your needs when assessing your students or employees:

As I’m writing this, I’ve just got back from the gym, where I beat my personal best distance on an exercise bike. What’s this got to do with computerized assessment, you might ask? Hear me out.

You’re probably familiar with norm-referenced testing and criterion-referenced testing :

• A norm-referenced test compares a test-taker against his or her peers. For example, you might compare my results with those of my Questionmark colleagues. (If you did, then seeing how energetic many are in the gym, I suspect my performance would not compare well!)
• A criterion-referenced test measures a test-taker against external criteria. For example, it might be that people of a certain age should be expected to reach a certain distance in a certain time on an exercise bike.

A third type is sometimes called ipsative assessment.

• An ipsative assessment in an education/learning context compares a test-taker’s results against his or her previous results. This is how I measure myself at the gym – I am pleased that I am doing better than I have before. I’m not worried if this meets some external criteria or if I’m better or worse than other people.

It’s very common to use criterion-referenced tests as computerized assessments because they help us measure competence. If you want to be sure that your employees know the rules, if you want to validate a pilot to fly a plane, or if you want to check that someone has understood training, a criterion-referenced test is usually the way to go.

 

But an advantage of ipsative assessment is that it measures progress and development – a test-taker can see if he or she is improving and whether or not he/she is taking advantage of feedback from previous assessments. Using ipsative assessment can help all test-takers improve: A weaker performer will be encouraged by seeing performance improvements over earlier attempts, and a stronger performer can be challenged to do better. This can deal with the risks of the weaker performer becoming demotivated from a poor test result and the strong performer complacent from a good one. Ipsative assessment can be used for objective measures (e.g. did I get a better score?) and also for more subjective measures (e.g. am I more confident about something?)...

 

...To learn more on ipsative assessment in education and learning, one resource is this study by Dr Gwyneth Huges of the Institute of Education. (As a heads-up, the term ipsative measure is also used in a different, technical way in psychological testing as a within-person measure.)

 

Expertise is built up by deliberate practice, and being tested can help identify where that practice is needed. I think it’s helpful for all of us to remember that progress and improvement is a useful thing to measure as well as achievement and competency.

Click HERE to read the post in its' entirety.

How to Write Effective Multiple Choice Questions

The CareerTech Testing Center has created, "The Secret of Writing Multiple-Choice Test Items." 

We believe these "secrets" can help any instructor or test developer, at ANY level of instruction, develop more effective test instruments. We hope you take a look and apply these methods to improve your test development process and be sure to tell us what you think.

You can download the document for FREE from Yudu and share it with your colleagues!

Enlarge this document in a new window
Publisher Software from YUDU

The Twelve Days of Testing

I thought I would share the Twelve Days of Testing since final exams are beginning and I obviously seem to be getting into the holiday spirit....

1. Create a successful testing environment
2. Follow the recommended time for administering tests
3. Establish rapport with examinees
4. Limit test anxiety
5. Provide test security and eliminate cheating
6. Understanding the numbers I, II, III, IV
7. Interpreting test scores
8. Analyze individual tests scores across all levels - instructor, district, and state
9. Provide positive reinforcement and build upon an examinee’s relative strengths
10. Remediate an examinee’s relative weaknesses
11. Implement improvements across all levels
12. Read and learn “The Secret of Writing Multiple Choice Test Items”

Happy Holidays! J.T.

Understanding the “Depth” of Testing

What do you think are good synonyms for the word “test?” Evil, horrible, despicable…should I add more? I get those comments and more from a wide variety of people and groups. I sometimes feel hated as I can almost feel the knives in my back, but to be honest with all of you, testing really is a good thing!

For testing to be considered a good thing, we have to consider much more than just the test score. Just as in the picture on the right, the “Test Score” is only the tip of the iceberg, underpinned by “Program Improvement,” “Instructor Improvement,” and at its greatest depth, “Individual Student Improvement.” Without the “Individual Student Improvement,” evaluation of the “Test Score” is not totally meaningful or useful.

The “Test Score,” or the tip of the iceberg, is actually comprised of many factors such as:

• Reliability and validity of the test itself
• Intellectual functioning
• Genetics programming
• Maturational status
• Environmental influences
• Personality, social, and gender influences
• Education and training
• Life and work experiences
• Testing environment
• Examiner characteristics

The score is also important to many different stakeholders and therefore, there are many reasons for why we test (educational, psychological, legislative, etc.) At the present time, most educational discussions seem to focus solely on accountability and there are justifiable reasons for that. As long as money is involved, there will always be that argument, but that isn't what I consider testing in its “purest” theoretical form (more about the “purest” form later).

Just under the waterline on the iceberg is “State Improvement” or program improvement. Test scores, when aggregated and analyzed across a state or geographical area, can lead to overall program improvement. In order to use scores in a valid way to make decisions about students or programs, we must also remember that we must clearly define and measure the psychological/educational constructs or traits that a test purports to measure. Once this is accomplished, there are several things that we can analyze at this level such as whether the curriculum matches up with the specified state standards and the high level of thinking in those standards? If not, should a different curriculum be used or should the existing curriculum be revised? Should professional development be implemented across the state to ensure that the standards are being adequately addressed?

The next thing we should consider is “Instructor Improvement.” Has the instructor addressed all of the standards during instruction or has the curriculum been fully utilized? Does the instructor utilize effective teaching methods? As long as the tests are aligned with the standards then the instructor can aggregate and analyze their own student’s scores and improve their own instructional performance. This level could also encompass the local school site and the administrator could make recommendations for improvement across grade-level or academic area.

We now find ourselves at the greatest depth of the iceberg…“Individual Student Improvement.” This is what I consider the “purest” reason for assessing students. First and foremost a test involves an "individual" and it is simply a point in time reference of a student's ability on a set of predetermined standards/objectives. The “individual” is where we, at times, tend to lose our focus, but we should be mindful that preparing a student for education, the workplace, and for life is our goal and the reason for instruction and assessment.

I think there are two primary views that most people consider when testing and I would hope that we consider both of these views and others when evaluating individual results:

One view …is to determine whether or not the students have mastered the objectives.

Another View …is to identify individual relative strengths and weaknesses and to help the student by building upon relative strengths and by remediating relative weaknesses.

In other words, how do we help an individual student become as prepared for their future as they can be? Testing can be a difficult thing for a student as the results will demonstrate the concepts that they could not recall, did not understand or did not know. Test anxiety can also be a factor that affects performance, but we should communicate that most students will have an area of relative weakness and that a test not only points out what they don't know, but what they DO KNOW. After all, even your best students may demonstrate an area of relative weakness. For example, if a student scored 94/100 on an exam, but every question missed was in a specific area, then we should never just hand the results back and say, "Great job!" We should say, "Great Job!' and use positive reinforcement as we address their area of relative weakness. Assessment, in an educational setting, is a form of important feedback and it should discriminate, not against individuals, but across variables as we seek to maximize their learning.

To briefly summarize, a student will find success on any educational test when their ability and effort are effectively combined with instruction and resources. This is really a fluid equation as one portion of this concept may be weighted more heavily than another and this may effectively offset a weakness in any portion of the equation. The key is to analyze the results from a number of different perspectives and to always ask, "WHY?" We should always analyze the results and use the information to modify our instruction according to a student's needs. We should never receive an individual student's test score and just file it away.

My main point is that many internal and external factors (home environment, socio-economic status, genetics, cognitive ability, testing environment, specific learning disabilities, etc.) contribute to a test score and we should always do our best to analyze the results for each individual student.

Our goal is to try and understand "ALL" of the factors that comprise an individual score and to maximize each and every student’s learning experience.

Instructional Design: Start With the Assessment!

I wanted to share a blog post with you written by Doug Peterson from Questionmark:

eLearning Design: Start with the Assessment!

I would venture to guess that many elearning designers/developers start designing their new elearning course “at the beginning” – they start writing content and gathering illustrations, then maybe work in some delivery considerations, go through the review and sign-off procedures … and only at the very end do they remember, “Oh, yeah, I should probably have some sort of quiz or test.”

Dr. Jane Bozarth is an accomplished elearning designer and developer, and her latest article in Learning Solutions Magazine is called Nuts and Bolts: The 10-Minute Instructional Design Degree. In her article she recognizes that a lot of elearning designers and developers come from other disciplines and may not have much formal training when it comes to elearning, so she provides eight recommendations for designing and developing the best elearning possible.

Her #1 recommendation? Design assessments first. Jane writes:

Too often we create assessments and tests as an afterthought, in a scramble after the training program is essentially otherwise complete. The result? Usually, it’s a pile of badly written multiple-choice questions. When approaching a project, ask: “What is it you want people to do back on the job?” Then, “What does successful performance look like?” “How will you measure that?” Design that assessment first. Then design the instruction that leads to that goal.

For example, I used to support the call center agent training for a large telecommunications company. It was important that the agents come out of the training with an understanding of the software applications they would be using at their station – not just the correct values for certain fields, but an understanding of the application itself, including how to log in, how to navigate, which fields were mandatory and which were not, etc. Therefore we knew we had to include software simulation questions in our assessments (something that can be done amazingly well with Flash in Questionmark Perception), which in turn meant that we knew we had to include simulations in our training.

Does this mean that your elearning will be “teaching to the test?” Some people might see it that way, but I would suggest that since the test reflects the desired behaviors back on the job, teaching to that test is not a bad thing. And by working backwards from the specific desired behaviors and the assessment of those behaviors, your training will be very focused on just what is needed.

I hope you will read all of Dr. Bozarth's article, "Nuts and Bolts: The 10-Minute Instructional Design Degree." The CareerTech Testing Center offers 100+ skills standards and assessments and we have long advocated that skills standards should be the foundation of instructional design and assessments. Whether it's K-12, Higher Ed., or CareerTech education, we are preparing students for the workforce. Successful job performance for our students is our ultimate goal and I think that Dr. Bozarth is correct when she states that we should, "Design that assessment first."

Intelligence in Teenagers Can Increase or Decrease Over Time

I always thought that intelligence was fixed, but apparently there is hope for teenagers (mine included).

Cognitive abilities can increase or decrease during the teenage years and it's important to keep children mentally active and off the couch. I've often "preached" how some people give up on certain kids, but these findings demonstrate how important it is that we challenge every student. Will this affect how you teach?



In the article, As Brain Changes, So Can IQ, published in the Wall Street Journal by Robert Lee Hotz, the author states, "the new findings by researchers at University College London, reported online in Nature, suggest that IQ, often used to predict school performance and job prospects, may be more malleable than previously believed—and more susceptible to outside influences, such as tutoring or neglect." As the article states:

A teenager's IQ can rise or fall as many as 20 points in just a few years, a brain-scanning team found in a study published Wednesday that suggests a young person's intelligence measure isn't as fixed as once thought.

The researchers also found that shifts in IQ scores corresponded to small physical changes in brain areas related to intellectual skills, though they weren't able to show a clear cause and effect.

"If the finding is true, it could signal environmental factors that are changing the brain and intelligence over a relatively short period," said psychologist Robert Plomin at Kings College in London, who studies the genetics of intelligence and wasn't involved in the research. "That is quite astounding.".....

...."An important aspect of the results is that cognitive abilities can increase or decrease," said Oklahoma State University psychometrician Robert Sternberg, a past president of the American Psychological Association who wasn't part of the study. "Those who are mentally active will likely benefit. The couch potatoes among us who do not exercise themselves intellectually will pay a price."

Another great resource on this subject is: Larry Ferlazzo's: The Best Articles On The New Study Showing That Intelligence Is Not “Fixed”

I hope you will take the time to research this subject further:
Nature: Verbal and non-verbal intelligence changes in the teenage brain
The Atlantic: Study of the Day: IQ Rises and Falls During Teenage Years
The BBC: IQ in teenagers 'can change more than thought'
Education Week: Study: Adolescents Can See Dramatic IQ Changes
Science: IQ Is Not Fixed in the Teenage Brain
Science: What Does IQ Really Measure?
The Los Angeles Times: IQ changes in teens as brain changes, study finds
The Wall Street Journal: As Brain Changes, So Can IQ

Satellite Tracking Systems Used to Prevent Exam Theft

Is testing a big business or what?

Bruce McDougall reports in the October 17, 2011 issue of the Daily Telegraph how the theft of a box of completed NAPLAN tests from a primary school in Australia caused a change in test security policy.

The article, entitled "War on HSC Cheats Beefed Up With New Satellite Tracking Technology to Prevent Exam Paper Theft" discusses how "The 13 million pages of exam papers and 1.75 million answer booklets will be sent to test centres across the state in hi-tech tamper-proof packaging with special bar codes that can pinpoint their location at any time." Exam authorities are also cracking down on cheating in the exam room and are requiring students to take a special course on ethics, plagiarism and copyright.

I think it is interesting how technology is being used to handle a "low" technology (paper and pencil) test.

Click HERE to read the article in its entireity.

What Makes a Good Diagnostic Question?

I just read a great post by John Kleeman on the Questionmark Blog that I wanted to share with you concerning what makes a good diagnostic question?

I think Kleeman's post provides further proof that assessments are a good thing and not an "evil" mandated by law. I spend a lot of my time defending assessments and I will agree that an assessment is "evil" when we assess just for the sake of assessing. ANY assessment should be built with a purpose and with much thought and the interpretation of results should be afforded the same treatment (of purpose and thought).

Assessments should be used to assist students with their individual relative strengths and relative weaknesses and to assist a teacher with their instruction or instructional materials. Please read the following post by John Kleeman:

What makes a good diagnostic question?

First, it should be almost impossible for someone to get the right answer for the wrong reason: A participant will only get the question right if he/she has the right idea about whatever the instructor wants them to be able to know, understand or do.

Second, wrong answers should be interpretable: If a participant chooses a particular wrong response, the instructor should be able to guess why the person has done so and what misconception he/she has.

So suggests Dr. Dylan William in his excellent new book, Embedded Formative Assessment (published by Solution Tree Press, and recommended). A common use for diagnostic questions is to find out whether participants have understood your instruction – telling you whether you can go onto another topic or need to spend more time on this topic. And if participants get it wrong, you want to understand why they have done so in order to correct the misconception. Good diagnostic questions involve deep domain knowledge, as you have to understand why learners are likely to answer in a particular way.

One tactic for creating diagnostic questions is to look at answers that students give in open-ended questions and choose common ones as distractors in multiple choice questions.

Here is an example of a multiple response diagnostic question quoted in the book:

There are 64 possible answers to the question; the right answer is B. and D. It’s pretty unlikely that someone who does not understand Pythagoras’ rule will get the question right, and if they get it wrong, there will be good clues as to why.

Questions like this can be hinge-points in instruction – they give you evidence as to what you do next. Do you need to continue instruction and practice on this topic, or can you move on to the next topic?

John Kleeman is the founder and current Chairman of Questionmark. He wrote the first version of the Questionmark assessment software system and founded Questionmark in 1988 to market, develop and support it. Kleeman has a degree from Trinity College, Cambridge, and is a Member of the British Computer Society and a Chartered Engineer. Having been involved in assessment software for more than 20 years; he has participated in several standards initiatives and was one of the original team who created IMS QTI. He also gathered valuable experience being the instigator and chairman of the panel that produced the British Standard BS 7988, which has now become ISO 23988.

New Legal Office Assistant Assessment and Skills Standards from the CareerTech Testing Center

I wanted to announce the new Legal Office Assistant skills standards and assessment being offered by the CareerTech Testing Center.

We are also very pleased to announce that the skills standards have been aligned with NALS...the Association for Legal Professionals.

The following duty areas comprise the skills standards (take a look at each duty area on the standards for the specific tasks within each duty area):

• Demonstrate Proper Use of Basic Legal Concepts and Terminology
• Demonstrate Grammar Usage Skills
• Demonstrate Appropriate Communication Skills
• Create Legal Correspondence
• Create and Format Court and Legal Documents
• Demonstrate Appropriate Mail Handling Procedures
• Demonstrate Appropriate Filing Procedures
• Demonstrate Knowledge of Basic Accounting Terms and Procedures
• Demonstrate Familiarity with Computer Information Systems
• Demonstrate General Legal Knowledge
• Demonstrate Knowledge of Code of Ethics and Professional Responsibility
• Demonstrate Time Management Skills

Please contact the CareerTech Testing Center at 405.743.5412 to order this new assessment! As always, you can access our skills standards for FREE on our website.

Visit NALS, the association for legal professionals. NALS remains a leader in the legal services industry offering professional development by providing continuing legal education, certifications, information, and training to those choosing the legal services industry as their career. NALS members represent every area of this industry from paralegals and legal assistants to legal administrators and office managers. Because of this diversity and an openness to welcome all members of the industry, NALS offers a broad spectrum of expertise to make the programs offered valuable to all members of the legal services industry. This allows NALS members to learn about other areas of the industry, making career enhancement as well as advancement easily attainable

How Do We Learn Best?...From Our Mistakes

I was talking with a few people this evening about work, education, families....basically life in general and now I'm sitting here watching a Texas Rangers vs. LA Angels of Anaheim baseball game (by the way, another nice win by the Rangers), but I keep reflecting on the evening's earlier conversations. Most of the stories that were told were hilarious mistakes of our youth and "other" people's misfortunes at work. I know that none of us would admit publicly to what we do wrong at work. I mean we do like to keep our jobs and our income, but most of us can talk about the mistakes of "others."

Again, all of these funny stories involved mistakes and it's amazing how we remember so many details about our own mistakes. So what is a "mistake" anyway?

According to Dictionary.com, a "mistake" is defined as:

An error in action, calculation, opinion, or judgment caused by poor reasoning, carelessness, insufficient knowledge, etc.

Guess what? The definition never states that any of us are stupid for making mistakes in life or in school. Mistakes are often what we learn best from and we should expect mistakes from our children and our students.

We need to create an environment for students where they have the freedom to make mistakes. For them to understand that this is just "life" and mistakes are what drives us to succeed. After all Thomas Edison and the Wright Brothers had a plethora of mistakes, but they never lost their drive to create and to succeed.

Mistakes are just a part of learning (an important part in my opinion).

I know you are wondering why a testing blog is rambling on about creating an educational environment that condones making mistakes. After all, we want our students to have perfect scores on every test don't we? In a perfect world, yes would be the answer, but most of us aren't perfect and mistakes happen. There are also many reasons as to why a student may or may not score well on any given assessment.

From a psychometric point of view, the scores from a standardized test should create a normal (bell-shaped) curve, but this rarely happens. As I have said before on this blog, a test is "one" point-in-time piece of useful information. The following quote will provide you a good description of what constitutes a test?

A standardized test is a task or set of tasks given under standardized conditions and designed to assess some aspect of a person's knowledge, skill, or personality. A test provides a scale of measurement for consistent individual differences regarding some psychological concept and serves to line up people according to that concept. Tests can be thought of as yardsticks, but they are less efficient and reliable than yardsticks, just as the concept of verbal reasoning ability is more complex and less well understood than the concept of length. A test yields one or more objectively obtained quantitative scores, so that, as nearly as possible, each person is assessed in the same way. The intent is to provide a fair and equitable comparison among test takers. Green (1981, p. 1001) as cited in Jerome Sattler's Assessment of Children, 1992.

As Green states, tests are less reliable that the measurement of length and we therefore need to use caution and consider all factors prior to placing a  formal or informal "label" on a child. What educators need to do more of is to use the data that we collect to help students by building upon their strengths and remediating their weaknesses. I think we all too often throw the scores of most tests into a student's file without taking the time to evaluate strengths and weaknesses.

So as we head into the last few weeks of school and all of the testing commences, I hope you will look at each student's end-of-year assessments and do what is best for each individual student. Please consider all environmental or socio-economic factors and remember that any decision about students should include norm-referenced tests, informal assessments, observations, and interviews.

A test means so much more than just a score.