Education On Demand..... Within Your Hands

Dr. Robbie K. Melton

I recently had the opportunity to meet Dr. Robbie K. Melton who is the Associate Vice Chancellor for Mobilization and Emerging Technology for the Tennesseee Board of Regents (mTBR) and a well known "Appologist."

Robbie oversees mobilization strategies and initiatives for the sixth largest statewide system of public higher education in the country: mTBR serves students enrolled in 46 institutions throughout the state (six state universities, 27 technology centers for technical/vocational education, and all of the state's 13 community colleges).

A significant part of her function is to study technology and its current and future application on education. More specifically, mTBR seeks new innovations in emerging technologies, social networking, gaming, simulations, amd virtual worlds for the purpose of increasing recruiting, retention, graduation rates and to improve teaching, learning, and workforce development. TBR creates teams that are composed of staff, faculty, students, administrators, and industry partners who are tasked with working on specific problems related to both mobile technology and its use in an education context.

Robbie and her staff are provided with the latest technology and a classroom evaluation is conducted to determine how students interact with the devices (the evaluation also includes general usability, ADA, durability, and security issues). Besides students, they provide support to instructors and staff as well. I think it's an interesting that Robbie noted that a "digital divide" exists between student and instructor, but it's typically the instructor that has the out-dated mobile device.

Now for the exciting part.... TBR maintains a mobile app resource center that points to extensive collections for education: 70,000 apps and counting. The site is easy to navigate with drop-down menus for the type of app, mobile device, academic area, workforce cluster, etc. The site also provides ratings for mobile devices and has a plethora of videos and presentations as additional resources.

Robbie is a dynamic speaker that has a passion for not only educational technology, but for it's application. I hate to say it, because I like my job, but I think Robbie has one of the coolest jobs in education. Don't you agree?

How Can You Improve a Student’s Test Performance?

The answer to this question can be quite complex and many issues may be out of the educator's control (home environment, socio-economic status, cognitive ability, specific learning disabilities, etc.), but I have tried to comprise a list that any educator can use to optimize a student's test performance.

Below are some great ways to improve your student's test performance:

Provide the best possible testing environment.

The environment should be such that participants can concentrate on their assessments with minimal distractions. Considerations regarding the testing environment include:

• Consistent/adequate lighting levels.
• Temperature at a comfortable level with proper ventilation.
• Space is quiet with minimal distractions.
• Participants should be asked to behave consistently (no eating, getting up and moving about).
• Avoid/delay the test administration when a participant appears hurried, troubled, or ill.

Adhere to the responsibilities of the Test Proctor (in a test center environment).

Testing should be "fair" to all participants (by limiting the ability to cheat and by providing accommodations to those with disabilities).

• Participant authentication: a picture ID should always be shown and login should be handled quickly and quietly by the proctor.
• Protection of the security of the online testing system. Username AND password should NEVER be revealed.
• Prohibiting the use of all communication devices (photos of test items and text messaging are common problems).
• Computer usage: Monitor whether participants are trying to access the internet or other programs.
• The proctor should be vigilant in their observance of the testing environment: Note passing, hand gestures, etc.
• Reference materials, texts, notes, etc., are not allowed in the testing area unless specifically allowed for in the exam or in a student’s Individualized Education Plan.
• If a candidate is caught cheating during an examination, testing will stop immediately. The candidate will receive a failing result and the incident will be reported.
• Students with an IEP may have special accommodations as specified in an IEP, IRP, 504, LEP, and ELL.

Timing is everything!

Ideally, students should be tested as soon as they have completed training and passed all skills performance evaluations. It is NOT a recommended practice to wait until the end of the academic year to test if the student is ready to test earlier.

Testing statistics prove that 70% of all certification exams are passed when students take their exams 3 to 7 days after course completion. This amount of time typically provides adequate study time and allows testing to take place while the information is still fresh. On the converse, the same statistics show over an 80% failure rate for students attempting their exam immediately after a class or if they wait more than 2 weeks after course completion.

Analyze test results.

Study the tests that you administer and learn to how correctly interpret the results to both students and parents. Whether its standard scores, T scores, age- and grade-equivalent scores, or percentile ranks, know your tests and know how to explain them in a manner that is understandable and with compassion! Every student is somebody's child and all feedback should be in a positive manner. Stress the strengths during your interpretation and remediate the weaknesses by building upon the strengths.

Results should be analyzed for each individual student (relative strengths and weaknesses), for an individual instructors and for the overall program. In other words, did an instructor adequately cover the standards? Did your curriculum align to the standards? Did the program meet your needs at the local or state level?

Develop and implement a remediation plan.

Relative strengths and weaknesses should be the focus of your remediation plan (Your test analysis will identify these factors for you.). A student may receive a 90% score on an assessment, but they still may have an area of relative weakness and this should always be addressed. You should place as much emphasis on this as you would on some that struggled on the assessment. The goal is to maximize every student's ability!

Your remediation plan should involve more than the student. You should also analyze the test results to identify any deficiencies in instruction (Were all of the educational standards covered during instruction?) or curriculum (Did our curriculum address each of the standards?).

Leap Motion: Changing the Way We Interact with Computers?

I wanted to share a new gesture-controlled system that I read about called Leap Motion.

I believe the creators have designed something that could change our interaction with computers on many educational and occupational levels. Here are a few statements from their website:

• For the first time, you can control your computer in three dimensions using just your hand and finger movements and Leap Motion enabled software.
• The possibilities are endless, really. Art. Healthcare. Engineering. Gaming.
• Set down that stylus. The Leap Motion controller lets you create onscreen artwork, including 3D images, without any other devices.
• Interacting with 3D modeling software has never been easier or more intuitive.
• Whether you're on a Mac or PC, now you can click, grab, scroll and pinch, using only your hands.
• Sign on the dotted line without ink or paper. Just point your pen and sign your name in the air.
• Gaming addiction? With the Leap Motion controller, now you can crush the enemy with your bare hands.
• Surgeons, leave your gloves on. The Leap Motion controller lets you control 3D medical data onscreen with a wave of your hand.
• The Leap Motion works on Windows Vista/7/8 and Mac OS X, with Linux being on the agenda.

I also believe that Leap Motion can have a significant impact on any student or individual that needs assistance with the development of fine motor skills.

Please watch the video below to find out more about Leap Motion or click HERE:

The 200 Best Apps for Special Education

Here is a great resource from Edudemic that states "if you are a special education teacher or are simply looking for an innovative way to reach a student... this is for you." I think it has a lot of useful apps and it narrows down the search when shopping through the app store.

Take a look at this Scribd document by Eric Sailers:
iPhone, iPad and iPod touch Apps for (Special) Education

What is Ipsative Assessment and Why Would I Use It?

I read a recent post on the Questionmark Blog by John Kleeman, the founder of Questionmark, that I wanted to share with you. I have long been a fan of ipsative assessment since it allows you to compare a test-takers results against their own previous results. This is a great method for assessment when you are dealing with individual students, students in special education, those involved in career and technical education, and employees. The basic question you are trying to answer is, "Are their skills better today than they were yesterday?"

Please take a few minutes to read the following post by Mr. Kleeman and think about your needs when assessing your students or employees:

As I’m writing this, I’ve just got back from the gym, where I beat my personal best distance on an exercise bike. What’s this got to do with computerized assessment, you might ask? Hear me out.

You’re probably familiar with norm-referenced testing and criterion-referenced testing :

• A norm-referenced test compares a test-taker against his or her peers. For example, you might compare my results with those of my Questionmark colleagues. (If you did, then seeing how energetic many are in the gym, I suspect my performance would not compare well!)
• A criterion-referenced test measures a test-taker against external criteria. For example, it might be that people of a certain age should be expected to reach a certain distance in a certain time on an exercise bike.

A third type is sometimes called ipsative assessment.

• An ipsative assessment in an education/learning context compares a test-taker’s results against his or her previous results. This is how I measure myself at the gym – I am pleased that I am doing better than I have before. I’m not worried if this meets some external criteria or if I’m better or worse than other people.

It’s very common to use criterion-referenced tests as computerized assessments because they help us measure competence. If you want to be sure that your employees know the rules, if you want to validate a pilot to fly a plane, or if you want to check that someone has understood training, a criterion-referenced test is usually the way to go.

 

But an advantage of ipsative assessment is that it measures progress and development – a test-taker can see if he or she is improving and whether or not he/she is taking advantage of feedback from previous assessments. Using ipsative assessment can help all test-takers improve: A weaker performer will be encouraged by seeing performance improvements over earlier attempts, and a stronger performer can be challenged to do better. This can deal with the risks of the weaker performer becoming demotivated from a poor test result and the strong performer complacent from a good one. Ipsative assessment can be used for objective measures (e.g. did I get a better score?) and also for more subjective measures (e.g. am I more confident about something?)...

 

...To learn more on ipsative assessment in education and learning, one resource is this study by Dr Gwyneth Huges of the Institute of Education. (As a heads-up, the term ipsative measure is also used in a different, technical way in psychological testing as a within-person measure.)

 

Expertise is built up by deliberate practice, and being tested can help identify where that practice is needed. I think it’s helpful for all of us to remember that progress and improvement is a useful thing to measure as well as achievement and competency.

Click HERE to read the post in its' entirety.

Inside Story Flashcards: "The World's Most Interesting Way to Learn Words®"

I have to admit that I've spent too much time on Inside Story Flashcards, but I'm hooked.

Intially, I was skeptical of their claim, “The world’s most interesting way to learn words,” but I found myself looking at word after word.

What makes these vocabulary flashcards so great is the accompanying pictures and the fact that you can click on a speaker icon for audio of the word. Students (or you) can choose words at four different levels of difficulty : Basic, Easy, Medium and Hard. Students can also choose to hide the definition if they are wanting to quiz themselves. They provide online flashcards and free printable flashcards for offline use. (Other flashcard sets are available for purchase.) The only issue I have with the site is that I want a search box so I can search for words instead of having to scroll through the list.

Inside Story Flashcards could be a great resource for ESL students and special education.

Dyslexie is a Typeface for Dyslectics

I found an interesting website and video that I wanted to share with you concerning dyslexia (a developmental reading disorder that occurs when the brain does not properly recognize and process certain symbols).

According to the site studiostudio, Dyslexie is a font that:

"is especially designed for people with dyslexia. When they use it, they make fewer errors whilst they are reading. It makes reading easier for them and it takes less effort. The Dyslexia font is used by several schools, universities, speech therapists and remedial teachers.

Independent research undertaken by the University of Twente, proved that the Dyslexia font improves reading results.

Research
The study at the University of Twente showed that people with dyslexia made fewer reading errors when they use the dyslexia font compared using standard font.

A part of the conclusion of this study is:
The people with dyslexia made fewer errors, than normal readers, on the EMT when using the font “Dyslexie”. This is an indication that reading with the font “Dyslexie” decreases the amount of reading errors."

I haven't read anything else on the "Dyslexie Font" but it would be interesting to know if it truly works???

Understanding the “Depth” of Testing

What do you think are good synonyms for the word “test?” Evil, horrible, despicable…should I add more? I get those comments and more from a wide variety of people and groups. I sometimes feel hated as I can almost feel the knives in my back, but to be honest with all of you, testing really is a good thing!

For testing to be considered a good thing, we have to consider much more than just the test score. Just as in the picture on the right, the “Test Score” is only the tip of the iceberg, underpinned by “Program Improvement,” “Instructor Improvement,” and at its greatest depth, “Individual Student Improvement.” Without the “Individual Student Improvement,” evaluation of the “Test Score” is not totally meaningful or useful.

The “Test Score,” or the tip of the iceberg, is actually comprised of many factors such as:

• Reliability and validity of the test itself
• Intellectual functioning
• Genetics programming
• Maturational status
• Environmental influences
• Personality, social, and gender influences
• Education and training
• Life and work experiences
• Testing environment
• Examiner characteristics

The score is also important to many different stakeholders and therefore, there are many reasons for why we test (educational, psychological, legislative, etc.) At the present time, most educational discussions seem to focus solely on accountability and there are justifiable reasons for that. As long as money is involved, there will always be that argument, but that isn't what I consider testing in its “purest” theoretical form (more about the “purest” form later).

Just under the waterline on the iceberg is “State Improvement” or program improvement. Test scores, when aggregated and analyzed across a state or geographical area, can lead to overall program improvement. In order to use scores in a valid way to make decisions about students or programs, we must also remember that we must clearly define and measure the psychological/educational constructs or traits that a test purports to measure. Once this is accomplished, there are several things that we can analyze at this level such as whether the curriculum matches up with the specified state standards and the high level of thinking in those standards? If not, should a different curriculum be used or should the existing curriculum be revised? Should professional development be implemented across the state to ensure that the standards are being adequately addressed?

The next thing we should consider is “Instructor Improvement.” Has the instructor addressed all of the standards during instruction or has the curriculum been fully utilized? Does the instructor utilize effective teaching methods? As long as the tests are aligned with the standards then the instructor can aggregate and analyze their own student’s scores and improve their own instructional performance. This level could also encompass the local school site and the administrator could make recommendations for improvement across grade-level or academic area.

We now find ourselves at the greatest depth of the iceberg…“Individual Student Improvement.” This is what I consider the “purest” reason for assessing students. First and foremost a test involves an "individual" and it is simply a point in time reference of a student's ability on a set of predetermined standards/objectives. The “individual” is where we, at times, tend to lose our focus, but we should be mindful that preparing a student for education, the workplace, and for life is our goal and the reason for instruction and assessment.

I think there are two primary views that most people consider when testing and I would hope that we consider both of these views and others when evaluating individual results:

One view …is to determine whether or not the students have mastered the objectives.

Another View …is to identify individual relative strengths and weaknesses and to help the student by building upon relative strengths and by remediating relative weaknesses.

In other words, how do we help an individual student become as prepared for their future as they can be? Testing can be a difficult thing for a student as the results will demonstrate the concepts that they could not recall, did not understand or did not know. Test anxiety can also be a factor that affects performance, but we should communicate that most students will have an area of relative weakness and that a test not only points out what they don't know, but what they DO KNOW. After all, even your best students may demonstrate an area of relative weakness. For example, if a student scored 94/100 on an exam, but every question missed was in a specific area, then we should never just hand the results back and say, "Great job!" We should say, "Great Job!' and use positive reinforcement as we address their area of relative weakness. Assessment, in an educational setting, is a form of important feedback and it should discriminate, not against individuals, but across variables as we seek to maximize their learning.

To briefly summarize, a student will find success on any educational test when their ability and effort are effectively combined with instruction and resources. This is really a fluid equation as one portion of this concept may be weighted more heavily than another and this may effectively offset a weakness in any portion of the equation. The key is to analyze the results from a number of different perspectives and to always ask, "WHY?" We should always analyze the results and use the information to modify our instruction according to a student's needs. We should never receive an individual student's test score and just file it away.

My main point is that many internal and external factors (home environment, socio-economic status, genetics, cognitive ability, testing environment, specific learning disabilities, etc.) contribute to a test score and we should always do our best to analyze the results for each individual student.

Our goal is to try and understand "ALL" of the factors that comprise an individual score and to maximize each and every student’s learning experience.

Intelligence in Teenagers Can Increase or Decrease Over Time

I always thought that intelligence was fixed, but apparently there is hope for teenagers (mine included).

Cognitive abilities can increase or decrease during the teenage years and it's important to keep children mentally active and off the couch. I've often "preached" how some people give up on certain kids, but these findings demonstrate how important it is that we challenge every student. Will this affect how you teach?



In the article, As Brain Changes, So Can IQ, published in the Wall Street Journal by Robert Lee Hotz, the author states, "the new findings by researchers at University College London, reported online in Nature, suggest that IQ, often used to predict school performance and job prospects, may be more malleable than previously believed—and more susceptible to outside influences, such as tutoring or neglect." As the article states:

A teenager's IQ can rise or fall as many as 20 points in just a few years, a brain-scanning team found in a study published Wednesday that suggests a young person's intelligence measure isn't as fixed as once thought.

The researchers also found that shifts in IQ scores corresponded to small physical changes in brain areas related to intellectual skills, though they weren't able to show a clear cause and effect.

"If the finding is true, it could signal environmental factors that are changing the brain and intelligence over a relatively short period," said psychologist Robert Plomin at Kings College in London, who studies the genetics of intelligence and wasn't involved in the research. "That is quite astounding.".....

...."An important aspect of the results is that cognitive abilities can increase or decrease," said Oklahoma State University psychometrician Robert Sternberg, a past president of the American Psychological Association who wasn't part of the study. "Those who are mentally active will likely benefit. The couch potatoes among us who do not exercise themselves intellectually will pay a price."

Another great resource on this subject is: Larry Ferlazzo's: The Best Articles On The New Study Showing That Intelligence Is Not “Fixed”

I hope you will take the time to research this subject further:
Nature: Verbal and non-verbal intelligence changes in the teenage brain
The Atlantic: Study of the Day: IQ Rises and Falls During Teenage Years
The BBC: IQ in teenagers 'can change more than thought'
Education Week: Study: Adolescents Can See Dramatic IQ Changes
Science: IQ Is Not Fixed in the Teenage Brain
Science: What Does IQ Really Measure?
The Los Angeles Times: IQ changes in teens as brain changes, study finds
The Wall Street Journal: As Brain Changes, So Can IQ

A Great Resource for Educational iPad Apps

Just wanted to share a blog called Apps in Education written by Gregory Swanson that seems to be a great resource for teachers. As Swanson states, "One of the hardest thing with using the iPad in the classroom is finding the time to go through all of the apps in the iTunes Store listed under the education banner. We have started to list some of the apps we've found under each of the Key Learning Areas."

Here are a couple of posts that I thought were interesting:

Assessment Apps by Vicki Windman (Check out the IEP Checklist)

10 Best Apps 4 Teachers

As always...let us know your thoughts!

Also, thanks to Tommi Leach at the The Career Advisor  Newsletter for letting us know about this site!

How Do We Learn Best?...From Our Mistakes

I was talking with a few people this evening about work, education, families....basically life in general and now I'm sitting here watching a Texas Rangers vs. LA Angels of Anaheim baseball game (by the way, another nice win by the Rangers), but I keep reflecting on the evening's earlier conversations. Most of the stories that were told were hilarious mistakes of our youth and "other" people's misfortunes at work. I know that none of us would admit publicly to what we do wrong at work. I mean we do like to keep our jobs and our income, but most of us can talk about the mistakes of "others."

Again, all of these funny stories involved mistakes and it's amazing how we remember so many details about our own mistakes. So what is a "mistake" anyway?

According to Dictionary.com, a "mistake" is defined as:

An error in action, calculation, opinion, or judgment caused by poor reasoning, carelessness, insufficient knowledge, etc.

Guess what? The definition never states that any of us are stupid for making mistakes in life or in school. Mistakes are often what we learn best from and we should expect mistakes from our children and our students.

We need to create an environment for students where they have the freedom to make mistakes. For them to understand that this is just "life" and mistakes are what drives us to succeed. After all Thomas Edison and the Wright Brothers had a plethora of mistakes, but they never lost their drive to create and to succeed.

Mistakes are just a part of learning (an important part in my opinion).

I know you are wondering why a testing blog is rambling on about creating an educational environment that condones making mistakes. After all, we want our students to have perfect scores on every test don't we? In a perfect world, yes would be the answer, but most of us aren't perfect and mistakes happen. There are also many reasons as to why a student may or may not score well on any given assessment.

From a psychometric point of view, the scores from a standardized test should create a normal (bell-shaped) curve, but this rarely happens. As I have said before on this blog, a test is "one" point-in-time piece of useful information. The following quote will provide you a good description of what constitutes a test?

A standardized test is a task or set of tasks given under standardized conditions and designed to assess some aspect of a person's knowledge, skill, or personality. A test provides a scale of measurement for consistent individual differences regarding some psychological concept and serves to line up people according to that concept. Tests can be thought of as yardsticks, but they are less efficient and reliable than yardsticks, just as the concept of verbal reasoning ability is more complex and less well understood than the concept of length. A test yields one or more objectively obtained quantitative scores, so that, as nearly as possible, each person is assessed in the same way. The intent is to provide a fair and equitable comparison among test takers. Green (1981, p. 1001) as cited in Jerome Sattler's Assessment of Children, 1992.

As Green states, tests are less reliable that the measurement of length and we therefore need to use caution and consider all factors prior to placing a  formal or informal "label" on a child. What educators need to do more of is to use the data that we collect to help students by building upon their strengths and remediating their weaknesses. I think we all too often throw the scores of most tests into a student's file without taking the time to evaluate strengths and weaknesses.

So as we head into the last few weeks of school and all of the testing commences, I hope you will look at each student's end-of-year assessments and do what is best for each individual student. Please consider all environmental or socio-economic factors and remember that any decision about students should include norm-referenced tests, informal assessments, observations, and interviews.

A test means so much more than just a score.

CareerTech Testing Center: J.T.'s Personal Favorites

I decided last night that David Letterman shouldn't be the only one that has fun making Top 10 lists, so here are my personal favorites from the blog ("Testing and Educational Posts" and "Educational Technology Posts"):             

Top 10 Testing and Educational Posts

1. Free Offer from the CareerTech Testing Center: "The Secret of Writing Multiple Choice Test Items"
2. 5th Grader's Moving Keynote
3. Why Do We Test Students?
4. Understanding the Numbers
5. Assessment Security and How to Reduce Fraud
6. Before He Cheats - A Teacher Parody
7. The Pygmalion Effect: Are You Guilty?
8. Shmoop: An Excellent Resource for Students and Teachers!
9. Test Anxiety
10. Response to Intervention

Top 10 Educational Technology Posts

1. A Brave New World-Wide-Web
2. ANIMOTO
3. Visual Thesaurus
4. Visual Dictionary Online
5. Wordle
6. When Creativity and Web 2.0 Meet
7. Blurb.com
8. The Khan Academy: Education to Anyone, Anywhere
9. Guidebook to Internet Searching
10. Wolfram/Alpha - A Computational Knowledge Engine

Let me know what your favorites are or if you have new ideas that you would like to see in the blog!

Thanks again for reading and as always, share the blog with others!  J.T.

Sir Ken Robinson: Do Schools Kill Creativity?

I wanted to draw your attention to a MUST SEE TED presentation by Sir Ken Robinson. He wonders why don't we get the best out of people?

Robinson argues that it's because we've been educated to become good workers, rather than creative thinkers. Students with restless minds and bodies -- far from being cultivated for their energy and curiosity -- are ignored or even stigmatized, with terrible consequences. "We are educating people out of their creativity," Robinson says.

I highly recommend taking the time to watch the video. Watch here:


You can also watch Sir Ken Robinson on a free webinar March 30th, Tuesday (5pm Pacific, 8pm Eastern, 11pm GMT): Sir Ken Robinson on The Element.

Top 10 Posts of 2009

Here are the top 10 posts for 2009.  I have to admit that there were a couple of surprises, but they all offer valuable information that I think you will enjoy if you haven't previously read them. Also, don't forget to search the "Archive" or "Labels" for other posts that might be of value to you.

1. National Consortium on Health Science and Technology Education
   The most popular post for 2009 discusses partnership between the National Consortium on Health Science and Technology Education (NCHSTE) (now called The National Consortium for Health Science Education) and the CareerTech Testing Center.
   
   
2. The Secret of Writing Multiple-Choice Test Items
   This post discusses how multiple-choice questions (MCQ’s) play an important role in testing and are among the most widely used of selection-type items on examinations today and how they can be designed to measure a variety of learning outcomes. This post has gained a lot of traction and has generated a lot of response for us.
   
   
3. Visual Thesaurus
   Visual Thesaurus is an interactive dictionary and thesaurus which creates word maps that blossom with meanings and branch to related words. Although I am surprised at how popular this post was, I think it is a great tool that many of you can use. Try it out!
   
   
4. The Pygmalion Effect: Are You Guilty?
   According to Wikipedia, the Pygmalion effect, (or Rosenthal effect, or Expectancy Effect), refers to situations in which students perform better than other students simply because they are expected to do so. It appears that this "concept" is controversial to some of you, but I think the Pygmalion effect does exist, at times, in regards to education and social class.
   
   
5. Mike Rowe, Dirty Jobs, and mikeroweWorks
   Mike Rowe, host of the Discovery Channel series "Dirty Jobs," has begun a campaign to celebrate the working men and women in this country. His website mikeroweWORKS, is a public awareness campaign designed to reinvigorate the trades, reinforce the importance of skilled labor, and draw attention to our crumbling infrastructure. Read our post to see the letter that he sent to President Obama.
   
   
6.  Response to Intervention
   This interview with Nancy Goosen, Director of Special Services, Edmond (Oklahoma) Public Schools discusses the Response to Intervention (RtI) model. RtI is a service delivery model that uses data to guide instruction and intervention at increasingly intensive levels. Ms. Goosen goes into great detail on RtI, its implementation and its successes.
   
   
7. Understanding the Numbers - III
   This post continues our discussion on statistics and the interpretation of ratio intelligent quotients, standard scores, stanines, and finally the relationship among derived scores.
   
   
8. Why Do We Test Students?
   After a few conversations I thought it was time to address why we test and there are typically many reasons for why we test (educational, psychological, legislative, etc.). At the present time, most educational discussions seem to focus solely on accountability and there are justifiable reasons for that. As long as money is involved, there will always be that argument, but that isn't the purpose of this discussion.
   
   
9.  Culinary Techniques - A New Culinary Arts Product from MAVCC
   MAVCC announced the release of Culinary Techniques, ©2009. Techniques, the third book in the Culinary Arts series, explores the essentials of food production, including tools and equipment, stocks, soups and gravies, salads, hors d’oeuvres, breakfast cookery, meats, desserts, garde manger, and much more. The curriculum is designed to give students a solid grounding in the fundamental skills needed for work in the commercial kitchen. The MAVCC Culinary Arts Series aligns with the American Culinary Federation competencies. Search MAVCC's online catalog for other great products!
   
   
10. PhotoFiltre - Free Photo Editing and Effects
    PhotoFiltre is freeware that I use quite often. Did I mention that it is FREE (you probably know by now what that means to me)!!! Take a look!
    

I hope everyone enjoyed the CareerTech Testing Center Blog in 2009. Here’s to finding more to talk about in 2010! Happy New Year everyone!!! J.T.

Reporting Test Results to Parents

As promised in last weeks post, Limitations of Test Scores, the National Association of School Psychologists created a handout entitled, "Using and Understanding Test Scores - A Hand-out for Teachers" by Andrea Canter, Minneapolis Schools and I wanted to share not only the limitations of test scores, but how test results should be reported to parents.

According to NASP, "Parents are often overwhelmed by the test reports they receive from school personnel. In order to help establish a true partnership between parents and teachers, it is essential that information about student progress be communicated clearly, respectfully, and accurately. Test results should be provided to parents using:

• Simple, clear language free from educational and test jargon
• Explanations of the purpose(s) of the tests used
• Brief descriptions of the test procedure(s)
• Scores appropriate to the test's purpose
• Clear explanations of the meaning of test scores
• Information about the limitations of the test scores
• Corroborating information from other sources
• Opportunities for parents to ask questions about scores, testing, etc.
• Information about how the test results will be used.

  Raw scores and percentiles, when appropriate to the tests used, are generally the most easily explained and familiar types of scores to report to parents. A visual reference may also be helpful, particularly in explaining the concept of "average." Numbers can be misleading and in some cases frightening; it is important for parents to understand the limitations of any test score and to receive a comprehensive picture of their child's performance, not just test results."
 
The following texts provide an overview of assessment issues for educators, including test scores:
Hamill, D. (1987) Assessing the Abilities and Instructional Needs of Students. Austin, TX: Pro-Ed.
Salvia, J. & Ysseldyke, J. (1988) Assessment in Special and Remedial Education, 4th Edition. Boston: Houghton-Mifflin 

Limitations of Test Scores

I found a hand-out from the National Association of School Psychologists entitled, "Using and Understanding Test Scores - A Hand-out for Teachers" by Andrea Canter, Minneapolis Schools and I wanted to share a part of it with you that explains the limitations of test scores:

"Scores can be useful, but must be interpreted carefully, keeping the purpose of the test in mind. A single test can only reflect a sample of skills; therefore, every score includes some degree of error in measurement - no score is absolutely accurate. Score are influenced by many factors, but particularly by the reliability and validity of the test. Reliable tests will yield consistent results if administrered many times. Valid tests actually measure and predict what they are intended to measure. For example, a valid reading decoding test will accurately distinguish students with strong and weak decoding skills; it will accurately predict which students will perform well in a phonics program and which students may have difficulty. We can place greater confidence in the scores provided by reliable and valid tests.

Many tests are of limited value when used with students who come from communities or cultures which are different from the White, middle class groups from which the scores were derived. Students who grow up in poor, rural or inner city communities, in ethnic minority families, or in families who primarily speak a language other than English are not typically well-represented in the standardization of tests, and scores obtained by these students often reflect some degree of bias.

Certain test scores are only useful for certain purposes, and not for others. Percentiles and standard scores, for example, are not intended to measure student growth. They only show relative standing compared to others, and this relationship does not necessarily change as students progress through the curriculum. Raw scores, such as words read correctlyor number of multiplication problems solved correctly, can be plotted over time to reflect progress.

Limitations of test scores can be minimized by using other types of information to describe student performance. Test scores alone should never be used to make placement or other serious educational decisions. Observations, school and health history, parent information, daily work samples, etc. should be used in conjunction with test scores.

Limitations of test scores can be further minimized by the use of qualified assessment personnel such as school psychologists to assist in the selection of tests and the interpretation of results."

The following texts provide an overview of assessment issues for educators, including test scores:
Hamill, D. (1987) Assessing the Abilities and Instructional Needs of Students. Austin, TX: Pro-Ed.
Salvia, J. & Ysseldyke, J. (1988) Assessment in Special and Remedial Education, 4th Edition. Boston: Houghton-Mifflin

I'll have another post at the beginning of next week that explains how test results should be reported to parents. J.T.

10 Questions Every Parent and Student Should Ask About Testing

I found a 2006 CNN article by Audrey Schewe entitled, "Cheat Sheet for Parents on Testing: Ask Right Questions to Understand Impact on Children, Schools." The last couple of pages were from the National PTA and it involved questions to ask before and after testing. I'll list them for you first and then provide a few comments:

Questions to Ask Before Testing

1. Which tests are given in my child's grade?
2. What are the purposes of these tests? What will these tests measure? How will the scores be reported?
3. How will the results of the tests be used?
4. What do you do to prepare children for the tests? What can I do at home to help my child prepare to take the test?
5. What happens if my child performs poorly?

Questions to Ask After Testing

1. What do the test results tell me about my child's strengths and weaknesses?
2. What can I do to help strengthen my child's skills and abilities?
3. Are the test results consistent with my child's performance in the classroom?
4. How does my child compare with his peers?
5. Is the test data used to help improve instruction? What, if any, changes will be made in the classroom instruction?

First of all, I believe that all of these questions should be asked by both parents AND students. I think that a student should always advocate for themselves and taking responsibility for their actions is an important first step for any individual.

The first questions I would add to the list of questions to ask prior to testing is "who" will administer the test and "who" will interpret the test? It is important that a person knowledgeable in assessment should administer and interpret the test. A standard score of 70 is not "like a C average" and unbelievably, I have heard that interpretation before. I would also ask what happens if my child, or I, performs successfully? Almost any type of test will demonstrate areas of relative weakness and areas or relative strength. Your child may make a score at the 95 percentile, but he/she may have a weakness in a particular area. Like I have stated before, a test is a point in time reference and a multitude of factors comprise this score. A person knolwedgeable in assessment should include these factors when interpreting the test.

The questions that are listed to ask after testing are all important. From my point of view though, I'm most interested in building upon strengths and remediating weaknesses. Use the test results to establish obtainable goals for the student and then monitor their progress and continue to raise the bar. Although it is interesting to compare yourself or your child to others, I really care about maximizing a person's abilities. Everybody has a strength, something that is unique to them, and it's exciting to watch them find success!  One last thing, besides using test results to improve classroom instruction, the results should be evaluated in regards to program improvement as well.

Remember to empower yourselves as a student or as a parent. Become active in your education...get involved and ask lots of questions! Asking questions is the best way to learn!!! J.T.

NoNotes.com: A Special Education Resource?

NoNotes.com is something that I'm really excited about, especially for Special Education purposes!!!


Have you ever been so busy taking notes during a lecture that you felt "lost" after the lecture? You took great notes, but you really weren't able to analyze what was being said or had a chance to ask follow-up questions? NoNotes.com could be your answer!

NoNotes.com is a student founded company who’s goal is to improve the way students throughout the world learn. Using NoNotes will allow you to focus on learning while in class rather than having to focus your attention on taking notes. The service allows you to sit back, learn, ask questions and make the most of your class time. Simply record the lecture, upload the recording to their site and they will provide a written transcript  (turnaround time is typically 1-3 business days). Prices start as low as $9.57/hr class.

I know I could have used this service when I was in college. To not only help with my own note taking ability, but to rely on someone more reliable than my "friends." Were you ever sick and asked a friend to take notes and you basically received nothing in return, except for a response like, "Dude, we really didn't talk about anything new." You later find out that you had a test review on the day you missed, because the next lecture you attended was actually a test! That's right, the instructor moved the test up in the schedule to the next class period instead of two class periods from my abscence (which made the syllabus...WRONG!). I know...I know... I should have chosen my friends more carefully and I didn't do so well on that test, but I'm still not over it!  Where was NoNotes.com when I needed it?

Now for what I think could be the BEST part of this site that NoNotes doesn't really even highlight....Special Education! Now what is there about special education that NoNotes could influence???  Two things: Specific Learning Disabilities and Limited English Proficient (LEP) and Immigrant Students.

Student's with a Specific Learning Disability typically have average to above average cognitive ability, but a discrepancy exists between expected and actual ability in reading, written expression, or mathematical calculations. Wouldn't NoNotes.com make a great resource for these students? Instead of taking notes, these students could spend their lecture time trying to process the information and asking questions of instructors to clarify their understanding of the material.

Now what about Limited English Proficient (LEP) and Immigrant Students? NoNotes can translate notes into several languages and wouldn't it be a great way for these students to learn if notes were provided in both English and the student's native language?

I hope you will take a look at NoNotes.com and let us know how you used the service and what your thoughts are!

Response to Intervention (RTI) and Mathematics

In January, I posted an interview with Nancy Goosen, Director of Special Services at Edmond Public Schools that focused on how they have implemented Response to Intervention (RTI) within their school system. (The above graphic is from NASDSE: National Association of State Directors of Special Education).

Today, I found an article in Education Week, "Response to Intervention and Mathematics" that I thought would be a great follow-up to our previous interview.

The article states, "The federal Institute of Education Sciences recently released a practice guide for educators who would like to know some of the best practices for using a response to intervention framework for math education.

(You can learn more about response to intervention as a whole here and here.)

The guide (pdf) does not offer an endorsement of any particular program. Instead, it talks about the features that a good response to intervention program should have.

For example, one recommendation is that at least 10 minutes of an intervention session should be built on reinforcing retrieval of basic arithmetic facts. Weak arithmetic skills makes it harder to teach students more advanced concepts, the guide said. The guide also includes a number of examples and sample questions, which should make it a good resource for teachers and trainers. "

Response to Intervention

An Interview with Nancy Goosen, Director of Special Services
Edmond Public Schools

Just like any other academic area, CareerTech programs serve a number of students that are on IEP’s (Individualized Education Program for children with disabilities). Nationwide the IEP subgroup makes up approximately 13% to 14% of the overall student population. Of the IEP student population, almost half of the special education population is identified as having a Specific Learning Disability (SLD). The President’s Commission on Excellence in Special Education reported in 2002 that over the last 10 years there had been a 36% increase in the SLD category population.

CTTC: Has Edmond Public Schools experienced the same trends in the identification of students with a SLD?
NG: Our numbers have been consistent with the national averages. We believe there are problems with the IQ/achievement discrepancy SLD formula and this revealed the current identification system was simply a “wait-to-fail” model. This means the younger children in grades K through 1st grade and even 2nd grade, rarely have been identified with this method. Therefore, it usually was at the 3rd grade level when a student was identified as having a significant discrepancy. The Commission’s report indicated the older students with reading disabilities respond poorly to conventional special education interventions, ultimately revealing few students exiting from special education or making significant improvement in their reading skills.

CTTC: How has your school system decided to respond to these findings?
NG: With the Response to Intervention (RtI) model. RtI is a service delivery model that uses data to guide instruction and intervention at increasingly intensive levels. RtI is designed to encompass all children in a school. Students can be identified in younger grades, even in Kindergarten, as eligible for special education services. By assessing all children in the elementary school the chances of catching all children with reading difficulties is significantly higher than the discrepancy model. RtI incorporates a frequent progress-monitoring component to help make decisions about changes in instruction, interventions and supports for students to match their specific needs.

Students who are determined eligible for special education services through the RtI model have been through intense interventions prior to the determination of IDEA (Individuals with Disabilities Education Act) eligibility. The students who are NOT determined to qualify for special education services under the SLD category have responded to the interventions implemented. Therefore, by implementing RtI model, districts should see a reduction in the SLD student population.

CTTC: How did you begin implementing RtI into your school system?
NG: We are piloting RtI in three buildings and making efforts to train staff to expand for next year. We hope to add two more elementary buildings next year. It is a huge change for our regular education teachers, so it takes quite a bit of training, but it provides opportunities for them to share their feedback to help make things run smoothly.

We conduct universal screenings three times a year and these screenings provide the local norms and we are also using national norms from AIMSweb. We then conduct progress monitoring with the bottom 16%ile. It has been a wonderful model, because our teachers are actually discussing specific interventions. No longer are they talking about reducing items or preferential seating, but asking questions of each other on how to teach a student struggling with Consonant/Vowel/Consonant (CVC) blends or how to teach specific concepts of punctuation. It is pretty cool to hear their discussions where they focus on specific interventions and what we can do to help the student.

CTTC: What is your goal for RtI?
NG: Our goal is to see the overall IEP subgroup reduce by half from 14% to 7% due to the increased intensive interventions implemented at an earlier age.

CTTC: What type of results have you seen to this point?
NG: We conduct universal screenings three times a year (fall, winter, and spring). From the universal screenings, progress monitoring is conducted with students falling at the 16%ile or below, which creates a group of students who received Tier 1 interventions. Approximately 10% of the student population tends to fall in this category of Tier 1.

Then when students do not respond to the Tier 1 interventions, they are moved to Tier 2, which tends to be approximately 7% of the student population. With intense interventions implemented at the Tier 2 level, it results in less than 1% of the total student population moved to Tier 3 receiving special education services as the intervention. The other 5% responded positively to the intense Tier 2 interventions without the need for special education services, and approximately 1% moved away during the middle of the intervention phase.

CTTC: Can you tell us a little more about the specifics of RtI? For example, what is meant by progress monitoring?
NG: Progress monitoring is the systematic and repeated collection of data to indicate the effectiveness of the intervention. The measures are administered frequently (often weekly) and assess growth in the skill being addressed by the intervention. An assessment system called Curriculum Based Measurement (CBM) is almost always used for progress monitoring.

CTTC: What is Curriculum Based Measurement?
NG: Curriculum Based Measurement, or “CBM”, refers to a collection of standardized procedures for collecting data on academic skills. CBM has 3 decades of research support in assisting educator’s in making sound, data-based educational decisions. Assessments such as DIBELS and AIMSweb are based on CBM procedures.

CTTC: You also discussed Universal Screening. Could you describe that process?
NG: Universal Screening refers to the screening of all children in basic academic skills. Data from the screening is used to identify students at-risk of academic difficulty, provide an indicator of the success of the core curriculum and instruction, and to provide data that is socially and culturally sensitive to the local school population.

CTTC: What is meant by “Tiers” in RtI?
NG: RtI uses multiple levels of intervention intensity referred to as Tiers. The most common model uses 3 Tiers in which the core curriculum and instruction is Tier 1, more intense intervention with additional resources is Tier 2, and prolonged, individual intervention is Tier 3. However, not all models are alike. Some may have 4 tiers or more, and some models include special education in a tier, and some do not.

CTTC: How do you determine cut-off scores or the number of students to conduct progress monitoring for Tier I, then Tier II?
NG: While there is no single method for determining cut scores, scores should predict risk of academic problems. A school site must also consider their capacity to provide interventions at each tier. RtI literature typically indicates that the core curriculum should be successful with 80-85% of students, additional intervention should be successful with an additional 10-15% of students, and the most intensive individualized interventions should only be necessary in approximately 5% of the school population. Using these projections as a guideline a school site might use the 20th Percentile as a cut-off. Some assessment packages currently available such as DIBELS, AIMSweb, easyCBM provide cutoffs. Local norms/ performance should also be considered.

CTTC: How long should you implement the interventions in Tier I and Tier II?
NG: The duration of intervention at a particular tier should be guided by data and should not be time limited. If an intervention is successful at a particular tier it could continue at that level until the problem is resolved. Tier timelines should be seen as the minimal amount of time prior to moving to a more intensive tier. Some general guidelines might include a minimum of 4 weeks at Tier 1, and a minimum of 8 weeks at Tier 2. Additionally, Tier 2 may require at least one intervention change prior to moving to the next tier.

CTTC: What type of interventions should be implemented?
NG: Interventions should be specific to the problem identified and be designed to increase competence. Interventions should be research based so that a particular outcome can be expected based on previous evidence. While interventions may be pre-packaged and commercially published, they may also be taken directly from the research literature provided they are described in sufficient detail. School should consider creating an intervention bank of protocols for the most frequently encountered problems.

CTTC: How do you determine when an intervention is not working and when do you change course?
NG: There is not currently a single prescriptive method used within all RtI models to determine an appropriate response to intervention; however, it is clear that implementing sites should clearly define their decision rules. It is also clear that the use of graphs is essential in making good decisions about student progress. Clear decision rules improve decision making. Assessments such as DIBELS and other CBMs have decision rules. Additionally, school districts have created decision rules using CBM data. For example, a district might determine that reaching the 25th percentile within 1 school year given grade level CBM represents a sufficient response to intervention.

CTTC: How are special education decisions made using RtI?
NG: With the passing of IDEA 2004, states may now allow schools to identify children with Specific Learning Disabilities using RtI data. If a student’s progress monitoring data does not indicate sufficient progress when provided with scientifically research based instruction and intervention, the team may determine the presence of a learning disability. An evaluation for special education eligibility, however, must be sufficiently comprehensive to rule out other factors.

As always, I think it is amazing to find out about some of the great things that are happening in the field of education. Intervention at an earlier age would surely reduce the frustration level for many students and increase their level of success. I would like to thank Nancy for all of the great information concerning RtI and I hope you will contact her or Dr. Scott Singleton if you have any questions!

If anyone would like to find out more about RtI, please contact:

Dr. Scott Singleton
Edmond Public Schools
Washington Irving Elementary
18101 N. Western
Edmond, OK 73034
Phone: (405) 340-2210

If you have any administrative questions, please contact:

Nancy Goosen
Director of Special Services
Edmond Public Schools
Phone: (405) 715-6116
Fax: (405) 330-7387
E-mail: nancy.goosen@edmondschools.net