Problem, Performance and Program in the A-Model

John Kleeman, the founder of Questionmark, poses the following question:

"How do you evaluate the effectiveness of training?"

Kleeman explains:

"Around 50 years ago, Dr. Donald Kirkpatrick wrote about what became known as the Kirkpatrick model – where you evaluate training in four levels: reaction, learning, behavior and results. His model has been hugely influential in the evaluation of workplace learning and the use of assessment.

This year, Dr Bruce C. Aaron, whose career spans the Florida Department of Education and working in evaluation at Accenture as well as volunteering at the chapter and national level with ASTD, has written about the A-model, a new way of thinking about assessment and evaluation of workplace learning.

The A-model does not start with the question: “How do you evaluate the effectiveness of training?” Instead, it puts the training in context, by starting with the 3 P’s: Problem, Performance and Program.

The basic concept is that in order to make sense of evaluating something, you have to start with the business Problem that you are trying to solve. Problems need to be important for the business purpose and they must be measurable – for example a need for increased production, improved quality, revenue increase, higher customer satisfaction.

Once you have a clear definition of the Problem (and how you will measure if it is solved), you need to define the human Performance that will solve the Problem. Performance itself will typically be behavior that is directly tied to important accomplishment or results in the workplace. To get performance, you use Performance Enablers – new learning of knowledge, new skills, changed attitude, feedback or incentives, performance support or new equipment. For example, you might identify that people need to be more skilled with a computer program to solve the Problem, or else to have better product knowledge.

Only after defining the Problem and the Performance, should you define the Program. The Program consists of the intervention or solution to the problem, and might be a training course, some other kind of learning or improvement or performance support.

Just as in the picture below, the Program is just the tip of the iceberg, underpinned by the Performance and the Problem. Without them, evaluation of the Program is not meaningful or useful.

Thus if your Program is a training course or learning intervention, you evaluate its effectiveness by measuring not only the delivery of Program itself but also determining whether it achieves the human performance improvement you are looking for and has a positive impact on the Problem it is there to solve.

A key difference between the Kirkpatrick model and the A-model is that the Kirkpatrick model starts out with the training course and goes on to tie it into results, whereas the A-model starts with a business problem and helps you define how to improve performance to solve the problem, and then measure training or other performance support in that context.

In Kleeman's next post on this topic, he’ll talk about how the A-model informs the processes of analysis, design and evaluation and where assessments fit in. In the meantime, you may want to listen to this 8-minute podcast from Dr. Aaron or read his thought-provoking A-model white paper available here (free with registration)."

I'll be posting more about the A-Model in the near future...