The latest edition of Chief Learning Officer magazine contains an interesting article, Time's Up (you can also read the article on the author's blog). It is about Donald Kirkpatrick's Four Level Evaluation Model that was first published in a series of articles in 1959 in the Journal of American Society of Training Directors (now known as T+D magazine).
The author, Dan Pontefract, sums up the article in his last statement, “Diverging from the cockroach, it's time for the learning profession to evolve.” While Dan's article is thought-provoking, I believe it misses out on two points, 1) since it is old, it's no good and 2) it has not evolved.
Old Does Not Mean Outdated
Interaction Design is closely related to our craft, training and learning. While the concept of interaction design has been around for ages, it was not formerly defined until 1990 by Bill Moggridge, co-founder of the Silicon Valley-based design firm IDEO. While it is one of the newer design professions, it still relies on older tools. For example, one of the tools used is the affinity diagram that was developed by Jiro Kawakita in the early 1960s. Thus, being old does not mean a tool should be extinct. If that was true, the cockroach would have disappeared millions of years ago, yet, because it has evolved, it has managed to survive... much to the disgust of anyone who has had their home invaded by them.
enso circle by Vibhav
“Nature itself is full of beauty and harmonious relationships that are asymmetrical yet balanced. This is a dynamic beauty that attracts and engages.” - Garr Reynolds
While people who have had their homes invaded by the cockroach look at them in disgust, entomologists look at them as one of the marvels of natures. Learning/Instructional Designers should not look upon our tools, such as ADDIE and Kirkpatrick's model as disgusting objects that have invaded our craft, but rather more as entomologists look upon the lowly cockroach—marvels of our craft that have survived the test of time.
The Evolution of Our Tools
Just as the ADDIE model has evolved over time, Kirkpatrick's model has also evolved. One of its main evolutionary steps was flipping it into a backwards planning model:
- Results or Impact - What is our goal?
- Performance - What must the performers do to achieve the goal?
- Learning - What must they learn to perform?
- Reaction - What needs to be done to engage the learners/performers?
While I blogged of this in 2008, Flipping Kirkpatrick, Kirkpatrick himself wrote of this several years earlier. This method align's with how Dan's article says we should start, “start with an end goal to achieve overall return on performance and engagement.” In addition, this in no way treats learning as an event, but rather a process. What is interesting, is how closely Kirkpatrick's evolved model fits in with other models, such as Cathy Moore's Action Mapping.
Like the ADDIE model, Kirkpatrick's model is often called a process model. However, this is only true if you blindly follow it. If you remove your blinders and study and play with it, it becomes a way to not only implement formal learning, but informal, social, and nonformal learning as well. For example, in step three, What must they learn to perform?, does not imply strictly formal learning methods, but rather any combination of the four learning processes (social, informal, nonformal, and formal).
How do you see our tools evolving?
5 comments:
Agreed. Many of our tools have not evolved much themselves - except within other evolving contexts and processes and how we now apply them. Thanks for your post!
The voices of Ponetfract and Kirkpatrick are very representative of competing positions in the learning profession. I’m not sure the polarization is serving us well. I like your analysis here. There are truths on both sides of the fence and the artificial dichotomy between formal and informal learning are broadening the gap. Some thought’s on that on my blog here:
http://bit.ly/gOuVfY
Can't say I agree with your points here. For one, "flipping" the Kirkpatrick model for use as a planning tool is not evolution. It's simply an alternative use of the same old system, and using it this way ensures that the very system itself will fail to evolve, as the whole process, from design of learning to assessment of effectiveness becomes a circular journey enclosed in the four levels model.
Dan Pontefract's point was not that the Kirkpatrick model was ugly or "disgusting", but that it has not evolved along with the learning profession. The model's continual focus on learning as an event, a point-in-time intervention, does not reflect the way we learn the vast majority of what we need to be effective in the workplace (or anywhere else, for that matter). The event-based approach to learning is at the core of the Kirkpatrick evaluation model, and until it evolves to more accurately reflect how learning and mastery happens, it will continue to have its detractors - and they will continue to have good grounds for criticism.
Having said that, thanks for adding to the debate and providing a vehicle for discussion of an important topic to all learning professionals.
Hi Glenn,
I'm not sure what you mean by it being an "event" as the model carries the learning from the learning, to the classroom, to the workplace, and finally to the impact it has on the organization. Thus I see it as being very much a process. I wrote more about this is a new post, Using Kirkpatrick's Four Levels to Create and Evaluate Informal & Social Learning Processes.
However, if you blindly follow the process, then yes, it probably does looks like a rigid event-based model. Thus it is up to the designer on how he or she was to follow it -- like a rigid old model stuck in time or a process that flows with the ever changing needs of training.
Claude Lineberry said something to the effect that 89% of ISPI presentations cite Tom Gilbert, but only 14.5% of the citers have read him. I think something similar holds true for Kirkpatrick. I don't consider the four levels as much of a model; I see them as a simplified analogy, a good-enough way to guide trainers and their clients toward bigger-picture thinking.
I believe an awful lot of formal training is, if not divorced from the job setting, clearly operating under a separation agreement. As you suggest, considering the four levels (like it? learned it? use it? got results?) has at least the potential to reconcile the two sides.
Alas, my experience over many years suggests a sizeable portion of in-house trainers use the four levels much as they use the nine-dot puzzle, the Maslow pyramid, and that business about remembering 20% of what you read but 40% of what you eat (or however it goes): reflexively, almost ritualistically, as part the standard ingredients in a recipe for cognitive meatloaf.
Post a Comment