11 Tips to Help Prevent Cheating and Ensure Test Security

I just read a post by Julie Delazyn on the Questionmark Blog that I wanted to share with you concerning cheating and test security.

We have previously covered many topis concerning cheating, but this post is a little different as it focuses more on the test developer and delivery system. Some of Julie's points can be used in a classroom setting so I hope you will take the time to read all eleven items. Here is Julie's post in its entirety:

With the summer behind us, it’s officially fall, and that means schools, colleges and universities have launched into a new academic year.
In this time of tests and exams, the security of test results is crucial to the validity of test scores. Today, I’d like to introduce 11 tips to help prevent cheating and ensure assessment security.

1. Screening tests – A small pre-screening can be administered to prevent people from taking an assessment for which they are not yet prepared.

2. Candidate agreements – Candidate agreements or examination honor codes are codes of conduct that a participant must agree to before they start an assessment . Candidate agreements generally are phrased in a personal manner ; the participant agrees by clicking on an ―OK‖ or ―Yes‖ button to the code of conduct for the exam

3. Limiting content exposure/leakage – In order to limit the amount of question content being shown to a participant at any given time, consider using question-by-question templates. These present questions one at a time to participants so that exam content is not completely exposed on screen.

4. Screening participants who achieve perfect scores – Many organizations will automatically investigate participants who achieve perfect scores on an assessment. Perfect scores are rare events, and could be attributed to a test-taker having had access to answer keys. The Questionmark Score List Report provides a fast and easy way to identify participants who obtain 100% on their assessments. An organization can then conduct an investigation of these participants to ensure that no suspicious behavior had occurred.

5. Verifying expected IP addresses – If assessments are to be taken from a specific location, often the IP address of the computer in that location will be known. Verifying expected IP addresses is a useful way to screen whether participants somehow took an assessment from an unauthorized location.

6. Reviewing time to finish information – The overall time it takes for a participant to complete an assessment can be a useful way to screen for suspicious behavior. If a participant takes a very short time to complete an assessment yet achieves a high score, this could be an indication that they cheated in some way.

7. Using Trojan horse or stealth items – Trojan horse or stealth items can be used to help detect whether a participant has memorized the answer key. Stealth items are inserted into an assessment and look just like the other questions, but they are purposely keyed incorrectly and one of the distracters is marked as the correct answer.

8. Post information that cheater prevention tactics are used – Inform participants that cheater -detection tactics are regularly employed. This can help to deter the low – motivation cheaters.

9. Proper seating arrangements for participants – Implementing a seating plan where participants are equally spaced, with limited ability to see another participant‘s screen/paper, is an import strategy.

10. Using unique make-up exams – When offering a make-up exam, make sure to administer it in the same strict proctored environment as the scheduled exam. Also, having another test form available specifically for make-up exams can lessen the risks of cheating and exposure for the actual large-scale exam.

11. Using more constructed response questions – Constructed response questions, like essay or short answer questions, provide less opportunity for participants to cheat because they require them to produce unique answers to questions.

Related posts:
Top 20 Resources for a Successful Testing Program
Five Tips to Prevent Your Student From Cheating
5 Keys to Creating a Successful Testing Program
Reporting Test Results to Parents
10 Questions Every Parent and Student Should Ask About Testing
Study Tips and Skills
Test Anxiety

How Can You Improve a Student’s Test Performance?

The answer to this question can be quite complex and many issues may be out of the educator's control (home environment, socio-economic status, cognitive ability, specific learning disabilities, etc.), but I have tried to comprise a list that any educator can use to optimize a student's test performance.

Below are some great ways to improve your student's test performance:

Provide the best possible testing environment.

The environment should be such that participants can concentrate on their assessments with minimal distractions. Considerations regarding the testing environment include:

• Consistent/adequate lighting levels.
• Temperature at a comfortable level with proper ventilation.
• Space is quiet with minimal distractions.
• Participants should be asked to behave consistently (no eating, getting up and moving about).
• Avoid/delay the test administration when a participant appears hurried, troubled, or ill.

Adhere to the responsibilities of the Test Proctor (in a test center environment).

Testing should be "fair" to all participants (by limiting the ability to cheat and by providing accommodations to those with disabilities).

• Participant authentication: a picture ID should always be shown and login should be handled quickly and quietly by the proctor.
• Protection of the security of the online testing system. Username AND password should NEVER be revealed.
• Prohibiting the use of all communication devices (photos of test items and text messaging are common problems).
• Computer usage: Monitor whether participants are trying to access the internet or other programs.
• The proctor should be vigilant in their observance of the testing environment: Note passing, hand gestures, etc.
• Reference materials, texts, notes, etc., are not allowed in the testing area unless specifically allowed for in the exam or in a student’s Individualized Education Plan.
• If a candidate is caught cheating during an examination, testing will stop immediately. The candidate will receive a failing result and the incident will be reported.
• Students with an IEP may have special accommodations as specified in an IEP, IRP, 504, LEP, and ELL.

Timing is everything!

Ideally, students should be tested as soon as they have completed training and passed all skills performance evaluations. It is NOT a recommended practice to wait until the end of the academic year to test if the student is ready to test earlier.

Testing statistics prove that 70% of all certification exams are passed when students take their exams 3 to 7 days after course completion. This amount of time typically provides adequate study time and allows testing to take place while the information is still fresh. On the converse, the same statistics show over an 80% failure rate for students attempting their exam immediately after a class or if they wait more than 2 weeks after course completion.

Analyze test results.

Study the tests that you administer and learn to how correctly interpret the results to both students and parents. Whether its standard scores, T scores, age- and grade-equivalent scores, or percentile ranks, know your tests and know how to explain them in a manner that is understandable and with compassion! Every student is somebody's child and all feedback should be in a positive manner. Stress the strengths during your interpretation and remediate the weaknesses by building upon the strengths.

Results should be analyzed for each individual student (relative strengths and weaknesses), for an individual instructors and for the overall program. In other words, did an instructor adequately cover the standards? Did your curriculum align to the standards? Did the program meet your needs at the local or state level?

Develop and implement a remediation plan.

Relative strengths and weaknesses should be the focus of your remediation plan (Your test analysis will identify these factors for you.). A student may receive a 90% score on an assessment, but they still may have an area of relative weakness and this should always be addressed. You should place as much emphasis on this as you would on some that struggled on the assessment. The goal is to maximize every student's ability!

Your remediation plan should involve more than the student. You should also analyze the test results to identify any deficiencies in instruction (Were all of the educational standards covered during instruction?) or curriculum (Did our curriculum address each of the standards?).

Testing Integrity: Issues and Recommendations for Best Practices

Recent news reports of widespread or suspected cheating on standardized tests in several school districts around the country have been taken by some as evidence that we must reduce reliance on testing to measure student growth and achievement. Others have gone even farther, claiming that cheating is an inevitable consequence of “high-stakes testing” and that we should abandon testing altogether. To be sure, there are lessons to be learned from these jarring incidents, but the existence of cheating says nothing about the merits of testing. Instead, cheating reflects a willingness to lie at children’s expense to avoid accountability—an approach I reject entirely.

– U.S. Secretary of Education Arne Duncan, The Washington Post, July 19, 2011

  The National Center for Education Statistics (NCES) has just released a report entitled, "Testing Integrity: Issues and Recommendations for Best Practice."

This report is part of a broader effort by the U.S. Department of Education to identify and disseminate practices and policies to assist efforts to improve the validity and reliability of assessment results. The report draws upon the opinions of experts and practitioners who responded to the Department’s Request for Information (RFI), the comments and discussions from NCES’ Testing Integrity Symposium, and, where available, policy manuals or professional standards published by State Education Agencies (SEAs) and professional associations.

The report focuses on four areas related to testing integrity:

1. The prevention of irregularities in academic testing;
2. The detection and analysis of testing irregularities;
3. The response to an investigation of alleged and/or actual misconduct;
4. Testing integrity practices for technology-based assessments.

The report is available:
http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2013454.

The National Center for Education Statistics is part of the Institute of Education Sciences, within the U.S. Department of Education.

10 Reasons Why Assessments Make the World a Better Place

Here is a list of ten reasons why John Kleeman, founder of Questionmark, believes assessment makes a difference to the world. I'll agree with Mr. Kleeman when he states, "These are reasons that make me proud to work in the assessment industry."

1. Assessments give equality of opportunity. Open public examinations let people from any parentage or origin show merit. Since their invention in Imperial China, exams have become a powerful way to give equality of opportunity to all.
2. Assessments make the world safer. Medical professionals, aircraft pilots, air traffic controllers, power station workers, and even anyone who drives a car, has to pass an exam. There are still tragedies, but we are all safer than we would be without assessments.
3. Assessments are the best way to measure knowledge, skills and attitudes. Maybe one day, brain scanners will read our minds, but until that day, assessments are the best way of quantifying what goes on inside our heads. As the scientist Lord Kelvin famously said: “If you cannot measure it, you cannot improve it”.
4. Assessments are the cornerstone of learning. Assessments give learners and instructors signposts and directions to learn better, and are needed for effective and efficient learning. See Questionmark’s white paper Assessments through the Learning Process (free with registration) for more on assessments and learning.
5. Assessments reduce forgetting. Machines remember things forever, but people forget. As I have shared in this blog, taking a quiz or test gives recall practice, which helps retain information for the future, and reduces forgetting.
6. Assessments are one of the few ways to be sure people really understand. If you need to be certain people understand something – business rules, legal policies your organization has to comply with or other critical information – a well-crafted assessment can check not just that people think they get it but that they really do.
7. Assessments give objective data. By using results from surveys and other assessments, we can avoid subjective “gut” decisions, and gather objective information to make sound data-driven decisions, which use resources wisely.
8. Assessments define standards. By requiring a certification or entrance exam or other assessment criteria, we define standards for what people need to learn and do, and if these standards are strong, they encourage good behaviour.
9. Passing an assessment makes people feel good about themselves. It’s a great feeling to pass an exam and to know that you have demonstrated your competence and capability. Fair, valid and reliable assessments contribute to happiness.
10. Online assessments give access for all. Last but not least (as this was the key reason I decided to devote my working life to assessments), the ability to take surveys, quizzes, tests and exams at a distance gives access to education, work and development opportunities to all. We empower those who are geographically remote or too poor to travel, as well as disabled people and those with family commitments.

Free Gift: The Secret of Writing Multiple Choice Test Items

One of the things we have learned over the years at the CareerTech Testing Center is that Subject Matter Experts are definitely experts in their field, but many aren't aware of how to design an effective test question. Therefore, we created one of our most popular posts, "The Secret of Writing Multiple-Choice Test Items." (This post led to the creation of the FREE GUIDE that you can can access below.

We believe that these "secrets" can help any instructor or test developer, at ANY level of instruction, develop more effective test instruments. We hope you take a look and remember... It's FREE!
(Download free copy from Yudu link below)

Click to launch the full edition in a new window
Digital Publishing with YUDU

Study: Tech Classes Correlate with Better Achievement

I found a great article on eSchool News written by Laura Devaney that I wanted to share with you. I think it demonstrates how career and technical education (CTE) has evolved from the industrial era "vocational education" into a more robust training that integrates academic, employability, and technical skills that are in demand in today's marketplace. Please read the following excerpt:

High school students in Florida who took at least one technology course and industry certification exam had higher attendance rates and GPAs, on average, than students with similar backgrounds who did not take such a course, a new study finds (“Student Performance in Career and Technical Education,” conducted by Grunwald Associates with support from Adobe.)

Just what this means is unclear, but the researchers who conducted the study surmise that students who take technology classes preparing them with real-world skills might be more engaged in school.

I believe "engagement" is paramount to anyone's success and CTE truly answers the "WHY are we having to learn this?" question that so many students seem to have have. Certification is the other key component to this research study as it also demonstrates to the student why they are learning certain concepts, no matter how abstract, because they are now learning to meet industry standards in a career field that they have interest in. Certification bridges the gap between learning and the workplace and I believe this is why CTE engages students and I also believe that many people fail to understand this key component that exists in career and technical education.

Please click HERE to read the article (you will have to register to read all of it or you can click on the actual study shown above).

Do Certification Exams Give Business Benefit?



John Kleeman

 I saw an interesting post on Questionmark's blog by John Kleeman that I wanted to share with you:

Increasing numbers of technology vendors run certification programmes to help customers, employees and partners demonstrate competence in using or advising on the vendor’s technology. This is common in IT, in medical equipment, in the automotive industry and in many other high-tech industries.

Certification is an area where all stakeholders seem to be “winners”:

• Vendors who set up certification programmes gain by being able to define the skill sets and knowledge that people deploying their technology need and encouraging stakeholders to develop the knowledge and skills and so deploy the technology more successfully for customers.
• Participants benefit from certification as a way to learn and develop skills and demonstrate their competence, and it often helps in their career path.
• Customers and users of the technology benefit from more effective deployment by being able to ensure the skills of experts deploying the technology and being more likely to get a successful implementation.
• Employers of test-takers gain from their employees being more capable.

But how real is the benefit? How can you know if a well-designed and well-implemented certification programme will lead to improved performance?

There is some powerful evidence about this from an IDC study a few years back as reported on by Network World. This study looked at the benefit of certifications within IT network administration – surveying more than 1,000 IT managers. You can see some of the results in the chart below. For instance on average, unscheduled network downtime was about 20% lower at organizations with more certified IT staff.

This study related to one particular field of IT, but it seems likely that in any technical field, providing you follow good practice in developing your certification programme, similar results should apply. Therefore certification is likely to provide material business benefits.

I believe Kleeman's most important point is when he discusses the importance of certification on vendors, participants, customers, and employers. Measuring a construct is certainly complex, but what it boils down to is ensuring that the construct is being measuring in a valid way and then reporting/communicating that process to stakeholders (this is where the "value" of an assessment is added to all stakeholders).

The CareerTech Testing Center's competency assessments are used to help all stakeholders become "winners" and they have been providing value to students, instructors, and industry for over twenty-five years. Our assessments are used to evaluate student performance and the results reports communicate competency assessment scores to students and provide a breakdown of assessment results by duty area from within the skills standards. The results breakdown shows how well the student has mastered skills needed to perform major job functions and identifies areas of job responsibility that may require additional instruction and/or training.

Group analysis of student results also provides feedback to instructors seeking to improve the effectiveness of career and technology training. Performance patterns in individual duties indicate opportunities to evaluate training methods and customize instruction.

Please contact us at 800-522-5810 ext. 403 to find out more about our assessments and services.