Tuesday, November 30, 2010

The Sigmoid Curve, Personal Learning, and the "Business" of Education

I have been thinking the last couple of weeks about my own personal learning habits, the CareerTech Testing Center, and on the "business" of education.

The last few days I began to think of Charles Handy and his Sigmoid Curve. This S-shaped curve can be used to describe the life-cycle of products, organizations, and even relationships. As the curve symbolizes the fact that nearly all of life’s endeavours start slowly, dip and falter through an experimental stage before rising to a pinnacle of success, after which there is inevitable decline.

To avoid such decline, decisions have to be made about further improvement at the point where success is still growing and before the entity/individual starts to experience this plateau. For many, this is a difficult thing to do. After all, if you have just survived the difficult and trying times, who wants to begin new sacrifices and give additional effort? Shouldn't there be plenty of time to relax and bask in your success?

I think good leaders and good learners know that this is the time to think about the next phase of change; resting on laurels will invariably lead to decline.

On its own, the S-shaped curve is a kind of depressing, and not particularly helpful. Is my life and/or my organization on the downward curve? What's left? Just thinking of putting your head in the sand and waiting for the inevitable end?

Actually, the power of the Sigmoid Curve comes when you begin to add a second curve to the original curve (see below). Charles Handy suggests that constant growth and development is achievable if we start a new initiative before the first one begins to decline (point X).

Paradoxically, this means making changes when the first curve is nearing its peak and the venture is flourishing. This is when an organization has the time, resources and the energy needed to see a new curve through initial explorations and floundering. Although there will inevitably be more motivation to change at “Y” when the first curve is in decline, at that point it takes enormous effort to move to where one ought to be on the second curve.

How do you make the second curve happen?

Knowing when to start a new curve is one thing and getting it started at the right time is quite another thing. Whether the second curve is a new product, a way of operating, a strategy or a culture, it will require fresh ideas and inspiration; and always be different to the first.

Handy believes that in an organization, the people who lead the second curve will also have to be different. Not only do original leaders need to keep the first curve going while the second takes off, they will also find it difficult to abandon their first curve while it is doing so well: there is a strong temptation to recapture past glories. What this means is that for a time there will probably be tension, confusion, chaos, backstabbing, anarchy (the area between X and Y) while new ideas and new people coexist with the old until the handover between first and second curves is complete. I recommend that everyone be as open as possible and just communicate. Develop feelings of mutual trust.

As Handy puts it, the paradox of success is that what got you where you are won’t keep you where you are.

When do we begin the second curve?

The key question becomes, “where are we on the first curve, and when do we need to start the second?” Handy suggests we will only know this for sure when we look back and, without hindsight, it is best to proceed by guess and assumption.

What this means in practice is that we must constantly engage in second-curve thinking. We need to stay sceptical, curious and inventive, challenging the assumptions underlying our current curve and developing alternatives. We need to ask questions like, “if we did not exist, would we reinvent ourselves and, if so, what would we look like?”. It is this kind of thinking that gives birth to second curves.

I think this is a different way of thinking for many of us and in reality, we should celebrate different points of view and different personalities. Our culture should be one that creates an attitude of messing with success.

In summary, the message of the Sigmoid Curve is that we need the foresight to start making changes even when it is not yet obvious that change is necessary, and the courage to switch from one curve to the next when the time has come.

In these trying economic times, the management philosopher thinks he can survive -- even prosper -- in the tough new downsized world as long as he understands the forces that are shaping it.

Now I ask that you will examine your personal Sigmoid Curve for learning and also where your organization (school/Agency/business) is on the curve?

Most importantly, remember to:
  • Courageously ask the tough questions.
  • Cultivate an attitude of messing with success.
  • Creatively experiment with new combinations of old ideas.
  • Celebrate different points of views and personalities.
  • Encourage and celebrate new ideas.
  • Remember that contraries are your friends and not your enemies.
Adapted from Handy, C. (1994) The Empty Raincoat, London: Random House

Sunday, November 21, 2010

Effectively Communicating the Measurement of Constructs to Stakeholders

I wanted to share the following article written by Greg Pope, Analytics and Psychometrics Manager for Questionmark, and Kerry Eades, Assessment Specialist for the Oklahoma Department of Career and Technology Education. Both authors share an interest in test security and many other topics related to online assessment.

Greg Pope
There are many mentions on websites, blogs, YouTube, etc. about people (employees, students, educators, school administrators, etc.) cheating on tests. Cheating has always been an issue, but the last decade of increased certifications and high-stakes testing seems to have brought about a significant increase in cheating. As a result, some pundits now believe we should redefine cheating and that texting for help, accessing the Web, or using any Web 2.0 resources should be allowed during testing. The basic idea is that a student should no longer be required to learn “facts” that can be easily located on the internet and that instruction should shift to only teaching and testing conceptual content.

Kerry Eades
There are many reasons for testing (educational, professional certification and licensure, legislative, psychological, etc.) and the pressures that stakeholders feel to succeed at all costs by “teaching to the test” or to condone any form of cheating is obviously immense. Those of us in the testing industry should, to the best of our ability, educate stakeholders on the purpose of tests and on the development and measurement of constructs. Having better informed stakeholders would lessen the “need” and “excuses” for cheating and improve the testing environment for all concerned. A key element of this is promoting an understanding of how to match the testing environment to the nature of an assessment: it is appropriate to allow “open book” assessments in some cases but certainly not all. We must keep in mind that education, in general, builds upon itself over time, and for that reason, constructs must be assessed in a valid, reliable and appropriate manner.

Tests are usually developed to make a point-in-time decision about the knowledge, ability, or skills of an individual based upon a set of predetermined standards/objectives/measures. The “value” of any test is not only this “point-in-time” reference, but what it entails for the future. Although examinees may have passed an assessment they may still have areas of relative weakness that should be remediated in order for them to maximize their full potential as students or employees. Instructors should also observe how all their students are performing on tests in order to identify their own instructional weaknesses. For example, does the curriculum match up with the specified standards and the high level of thinking in those standards? This information can also be aggregated and analyzed at the local, district, or state level to determine program strengths or weaknesses. In order to use scores in a valid way to make decisions about students or programs, we must begin by clearly defining and measuring the psychological/educational constructs or traits that a test purports to measure.

Measuring a construct is certainly complex, but what it boils down to is ensuring that the construct is being measuring in a valid way and then reporting/communicating that process to stakeholders. For example, if the construct we are trying to measure in an assessment is “Surgery Procedure” and if the candidate passes the test, we expect that the person can recall this information from memory where and when needed. It wouldn’t be valid to let the participant look up where the liver is located on the Internet during the assessment, because they would not be able to use the Internet while they are halfway through a surgical procedure.

Another example would be “Crane Operation” knowledge and skills. If this is the construct being measured and it is expected that candidates who pass the test can operate a crane properly, when and where they need to, then allowing them to tweet or text during their crane certification exam would not be a valid thing to do (it would invalidate the test scores) because they would not be able to do this in real life.

However, if the assessment is a low stakes quiz that is measuring the construct, “Tourist Hot Spots of Arkansas,” and the purpose of the quiz is to help people remember some good tourist places in Arkansas, then an “open book” or an “open source” format where the examinee can search the internet or use Web 2.0 resources is fine.

Effectively communicating the purpose of an assessment and the constructs being measured by it is essential for reducing the instances of cheating. This important communication can also help prevent cheating from being “redefined” to the detriment of test security.

For more information on assessment security issues and best practices, check out the Questionmark White Paper: “Delivering Assessments Safely and Securely.”

(Click here to read the post on the Questionmark's blog.)

Thursday, November 11, 2010

Thoughts on Using Prezi as a Teaching Tool

Do you feel that slides limit your ability to develop and explain ideas? Then maybe it's time you try Prezi.

Prezi is a free, web-based, application which allows you to design non-linear presentations online. They offer a free version (and a fee-based version), it's simple to use, and you'll never need to create individual slides again.

Prezi allows you to zoom in and out, add graphics, and video(s) to your presentation. It’s difficult to explain how to use it so I recommend that you look at the example presentations and play with the system yourself. Prezi is different and so it does take some getting used to, but after some practice you will find that the user interface is quite intuitive. You can also find plenty of help, i.e. videos, forums, Twitter, and blogs, which will help you create your presentations.

Watch the Prezi below (or HERE) to get a perspective on how to use Prezi as a teaching tool:

Tuesday, November 9, 2010

DonorsChoose.org: An Online Charity That Connects You to Classrooms in Need

A teacher spends, on average, $40.00 per month on classroom essentials.

DonorsChoose.org is a site that can help teachers alleviate the burden of paying for school items out of their own pockets or to get funding for a new project. Not only can you submit a project request, but you can also choose a project to donate to. Whether you have $100.00 to donate or just a $1.00, no gift is too small.

DonorsChoose.org makes it easy for anyone to help students in need. DonorsChoose.org grew out of a Bronx high school where teachers experienced first-hand the scarcity of learning materials in our public schools.

Charles Best, then a social studies teacher, sensed that many people would like to help distressed public schools, but were frustrated by a lack of influence over their donations. He created DonorsChoose.org in 2000 so that individuals could connect directly with classrooms in need

According to the site:
Here's how it works: public school teachers from every corner of America post classroom project requests on DonorsChoose.org. Requests range from pencils for a poetry writing unit, to violins for a school recital, to microscope slides for a biology class.

Then, you can browse project requests and give any amount to the one that inspires you. Once a project reaches its funding goal, we deliver the materials to the school.

You'll get photos of your project taking place, a thank-you letter from the teacher, and a cost report showing how each dollar was spent. If you give over $100, you'll also receive hand-written thank-you letters from the students.

At DonorsChoose.org, you can give as little as $1 and get the same level of choice, transparency, and feedback that is traditionally reserved for someone who gives millions. We call it citizen philanthropy.
Take a look at DonorsChoose.org and see if you would like to fund a classroom project or submit a project proposal if you need help with funding.

What a great way to help a classroom during the holidays!

Friday, November 5, 2010

The Ultimate Twitter Guidebook For Teachers

Go ahead and admit it.  You always wanted to try Twitter, but you've never felt comfortable using new technology and you really couldn't determine how Twitter could be used in the classroom anyway. I mean who has the time during the school year to learn new technology?

The answer is quite simply....Yes, it's easy to learn and you do have the time because I've found the resource that you've been waiting for! The Ultimate Twitter Guidebook For Teachers (by EDUdemic) is a list of 100 tips, apps, and resources that are separated into the following categories:
  • Resources for Learning Twitter
  • Twitter for Educators
  • Resources for Making the Most of Twitter
  • Suggestions for Twitter Use in the Classroom
  • Apps and Twitterers to Use with Students
  • Apps to Make Twitter Work for the Educator
  • App Resources
  • Tweets to Follow
  • Fun Twitter Experiments
The upcoming holiday season would also be a great opportunity to try Twitter and begin a personal learning network. I hope you will follow the CareerTech Testing Center on Twitter @CareerTechTest

Contact me if you begin a new Twitter account (or even if have an existing account) because I would like to be a part of your learning network!

Wednesday, November 3, 2010

Diagnostic Tests That Measure Conceptual Understanding

I just read a very interesting post from Questionmark's founder John Kleeman. I think it's a great way to measure conceptual understanding and to identify misconceptions. I hope you will read John's post below and use the information to improve your assessments.
I’ve just read a thought provoking article on diagnostic tests written by Simon Bates and Ross Galloway from the University of Edinburgh Physics Education Research Group and published by the UK Physical Sciences Centre (see the article at pages 10-20 here).

The authors are particularly concerned with diagnostic tests that measure conceptual understanding and identify mis-conceptions. So rather than testing for facts or knowledge or particular skills, their interest in diagnostic assessments is primarily around whether students understand some key concepts in the Physical Sciences. If students don’t understand them, they as instructors need to correct this in their teaching and feedback.

The article gives examples of use of diagnostic tests and also gives some good and detailed guidance on how to construct them, including which statistics to look at for good results. They recommend (as proposed by other authors in the Physics Education Research literature) a p-value or difficulty index of 0.3 to 0.9, a discrimination index of 0.3 or better or a point biserial correlation of 0.2 or better, and a reliability index of 0.7 or better.

They also explain how to write questions that test why people don’t understand something as well as what they don’t understand. And they give the example below (from the Lawson Classroom Test of Scientific Thinking) as something they have used in their teaching. Here is a what-why question, which asks for a fact and also asks why that fact is the case.

Bates and Galloway report that the first, “what” part of the question is answered just as well by students coming into university as those who have completed their first year at university, but that there is significantly better performance in the “why” part by those who’ve been at university for a year.

Getting to the root of learner misconceptions is a key challenge for all of us in learning and assessment, and I recommend this article as a good read.

Monday, November 1, 2010

Are You Looking for Ways to Reduce Software Costs? Try alternativeTo

AlternativeTo is a new approach to finding good software for your computer or your mobile phone. Whether you're looking for an alternative to expensive commercial software or just trying to get a feel for what's out there, AlternativeTo is a handy service for finding software alternatives. Tell them the application you want to replace and they'll give you a list of great alternatives.

More specifically, alternativeTo allows you to sort programs according to operating system and user rankings and they provide notes about "questionable" software (one's that you should probably avoid). As always, I would hope that you would thoroughly research any software prior to using it anyway.

If you want an alternative to the "alternativeTo" try OpenSourceAlternative.
Related Posts Plugin for WordPress, Blogger...