If all you have is a hammer. . .

Then everything looks like a nice craft beer. . . or a nail:


Nowadays we expect so much more from our tools:


We expect them to be multi-functional. . . to include ADDIE and the Four Levels of Evaluations.

MERLOT 2013 Classics Award

I'm honored to announce that Big Dog's Leadership Page won a MERLOT 2013 Classics Award for developing an outstanding, peer reviewed online resource.



It is recognized as an exemplary model for business.

Flat Army: 3-33 Learning Model Verses 70-20-10

I have been reading Dan Pontefract's new book Flat Army: Creating a Connected and Engaged Organization, and can honestly say it is an excellent and insightful book for developing a collaborative and open leadership organization. The part that grabbed my attention the most is the section in chapter 9, Learning at the speed of need, which discusses the 70-20-10 learning model (while I have a great interest in all of Dan's topics in his his book, learning is my primary field of interest).

Dan notes that there is no empirical evidence that learning maps to the 70-20-10 model, even though practitioners often cite it as a fact. It was developed in the 1980s when command and control was at the heart of leadership—think of Lee Iacocca, Jack Welch, and Roger Smith who more than likely thought that Learning/Training Departments were solely for their bidding and could offer very little for them personally. Thus the model is based on the the very thing that many learning and development practitioners are trying to get away from—hierarchy organizations. In addition, the model was developed before the Internet, thus it does not account for the numerous technologies that have aided formal learning, such as just-in-time learning, elearning, virtual learning.

I would also add that while some have pointed to Where did the 80% come from? as further proof that the 70-20-10 learning model is valid, when I researched the listed references on the page that give low percentages to formal learning, such as Raybould who proclaims that formal learning only accounts for 10% of the learning, I discovered that the authors provided no evidence at all. Such citations seam to imply, “I saw it on the Internet so it most be true.” On the other hand, the references that provide higher ratios for formal learning are the most evidence based.

The 3-33 Model

Dan provides what I see is a closer approximation of the learning ratios: 3-33, which stands for 33% the learning is formal, 33% is informal, and 33% is social. What is most interesting is that the research behind his model revealed that when the learners were asked to give the percentages on how they thought they learned, the numbers were very different than when the researchers actually discovered how the learners did indeed learn. This coincides with other research that indicates what learners are able to judge about their learning experiences (see Learner Self-Assessment Ratings).

graphic by Dan Pontefract

One of the other major errors of the 70-20-10 model is that it places reading in formal learning. Since when did reading a book become formal learning? Dan of course places it under the correct type of learning in his 3-33 model... informal. The 70-20-10 error seems to again coincide with the command and control culture that was most prevalent in the 1980s—the top leaders viewed writers as part of the elite who they could trust and learn from, while the learning/training functions were viewed as something to command and control, rather than trusted partners.

All in all, Flat Army is a very good book that deserves a special spot in in anyone's library who is interested in collaboration, leadership, and learning.

DISCLAIMER: While Dan and I have exchanged comments via blogs, I have no other interests in the book.

NOTE: While I seem to be kind of harsh on “command and control” it is mostly because of the audience I write for. I'm retired military so my real view of command and control follows the military view, which vastly differs from layman's and others point of view. See my writings on Leadership, Management, Command, & Control.

What are your thoughts on the two models?

In Memory of My Loving Wife

As you probably noticed I have not wrote any new blog posts or tweeted for quite some time. My wife passed away last November. I knew this would come sooner or later as she had an advance stage of cancer that was diagnosed in January of last year. I thought I was better prepared, but Kyong and I have been married for over 39 years and soul mates for over 40 years  (it was both our first marriage). Thus it came as a bigger shock than I thought it would.

Kyong was willing to fight the cancer as long as she could maintain a certain quality of life, but said when it was time to go, she wanted to go. On November 6th she came down with pneumonia and was taken to the hospital with a low blood oxygen  level. They gave her forced oxygen and antibiotics, which quickly brought her oxygen level up to normal; however, the next day it started dropping again to an unsafe level and we decided it was time to put her on comfort care. Our daughter and I were both at her side when she passed peacefully away.

I had to drop off the virtual social grid as I was quite busy taking care of her, our home, and running my business during the last few months of her illness and then it took me quite some time to get over the grief of losing her. It's time to reenter the grid.

ADDIE 3.0

After seeing the title you might be wondering, “I didn't even know we had an ADDIE 2.0???”

When the first version of ADDIE appeared in 1975 it was strictly a linear or waterfall method. The first four phases (analysis, design, development, and implementation) were to be performed in a sequential manner. This is a good method if you are trying to prove something in that it helps to ensure that all the variables are accounted for. However, the majority of learning designers simply want to build a great learning process, and for that a more dynamic method is required that allows them to change and improve the learning process as they advance through their designs. Learning designers who were not locked in to processes improved upon ADDIE by making it an iterative model. Thus by the mid-eighties ADDIE became a dynamic model (U.S. Army, 1984).

As van Merriënboer notes, “The phases may be listed in a linear order, but in fact are highly interrelated and typically not performed in a linear but in an iterative and cyclic fashion.” This major improvement became ADDIE 2.0 in that it allowed designers to work in a more natural fashion.

van Merriënboer (1997) also noted another major improvement—other components may be added to it on an as-needed-basis. This greatly improved the versatility of ADDIE in that while it is a broad scope model that covers the basics of good learning deign, it fails to cover many of the details. Thus rather than being a stand-alone model, it is used with other design models. Thus it became ADDIE 3.0 in that ADDIE is used to guide the essentials of the design, while other models are used in conjunction with it to expand and improve the design methodology:

Some example include:

• Analysis - Complex problems can be difficult to identify by standing on the outside, thus you might need to jump into the problem itself. This is known as “Immersion” in Problem X Design. Another method is using narratives by having the customers tell stories of the problems they have faced. It often only takes a few stories to recognize a common theme that prevents them from higher levels of performance. This technique is used in System Thinking Design.
• Design - The 4C/ID model shows two basic approaches for presenting content and two basic learning strategies that gives us four instructional design methods that vastly improves upon the common method of simply presenting content to the learners.
• Development - Using backwards planning with Concept or Action Mapping to keep the goals of the learning process aligned with the business objectives.
• Implementation - Always consider other performance methods, such as performance aids, before deciding upon classroom learning.
• Evaluation - Flipping the Four Levels of Evaluation into a more effective model.

Updated Table of Five Design Models

I recently added a new row to the table of design methodologies that shows some examples after a request from a reader: A Table of Design Models: Instructional, Thinking, Agile, System, or X Problems?


I would be interested to know if the examples are helpful.

A Table of Design Methodologies: Instructional, Thinking, Agile, System, or X Problems?

With so many design models to choose from, what should I use? To help with the answer I created a table of five design methodologies that may assist you in choosing a model to start from:

• Instructional System Design
• Design Thinking
• Agile Design
• System Thinking
• X Problems

The table includes their definitions, visual models, primary focus and goals, values, main steps, and further readings (web links). Note that models are only guides on the sides, not sages on the stages, so don't hesitate to mix, match and adapt to help you arrive at a perfect solution to a difficult problem.
Let me know what you think.

Mapping Pedagogies For Performance

Clark Quinn wrote an extremely interesting post, X-based learning: sorting out pedagogies and design, on activity based learning. Wanting to see how these different models would interconnect on a mindmap, I started playing with them. That is when I noticed that one of the main difference among them is that some have a known answer and/or the goal is driven by the curriculum, while others have an unknown answer and/or the goal is directed by the learners.

It then struck me that the two primary branches should (could?) be the two main types of knowledge—explicit and tacit:

• Explicit Knowledge is normally easy to articulate to others, thus the models with known answers and/or driven by the curriculum would fall on this side of the branch.
• Tacit Knowledge is normally difficult to articulate to others, thus the models with unknown goals and/or directed by the learners would fall on this side of the branch.

This seemed to give the mindmap a real purpose, rather than just be formal vs. informal, social vs. self, or active vs. passive. Thus the map goes beyond activity based models:

For a larger map click on the image or here.

(note that you can hover your mouse pointer over each concept in the large map to learn more about it)

I'm not sure if I have all the concepts aligned correctly, thus I am wondering what your thoughts are?

Note: I used FreeMind (free of course) to create the mindmap. The document for the mindmap is here - Learning.mm - if you want to download it and revise it. If you have trouble downing it, this is the directory of all the files used to create the mindmap, pictures, and html file - http://nwlink.com/~donclark/learning/pedagogies/. Right click on the file you want to download.

ADDIE is the Scavenger of Instructional Design, Not the Bitch Goddess (or Blooming Beyond Bloom)

When ADDIE was first handed over to the U.S. Armed Forces it was a linear model. However, after working with it they found that they needed a more dynamic model, so they adapted it. They mastered the tool rather than become a slave to it.

For some reason instructional designers love building ADDIE into a goddess that orders them to build crappy learning platforms. For example, they pronounce that it only builds courses when the real fact is that it tells you to use a course only if a simpler method, such as a performance support tool or OJT, will not work.

From its inception, ADDIE was designed to be a lean, mean, instructional design machine. This leanness has fooled others into thinking that it is a universal model that can build strip malls and skyscrapers. Nope! ADDIE has specific steps that are strictly designed for learning. This has led others to believe that ADDIE is too lean, that it tells them what to do, but not how to to it. But as Merriënboer noted, you can add other components on to it when needed

ADDIE is a Scavenger, not a Hoarder

One of the learning tools that is perhaps most often plugged into ADDIE is Bloom's Taxonomy. And of course one of the criticisms often leveled at ADDIE is that it is associated with outdated learning models. However, this plug and play feature of ADDIE does not mean it hangs on to outdated models, but rather it sheds them and goes scavenging for a better one. While Bloom's Taxonomy has been quite useful in that it has extended learning from simply remembering to more complex cognitive structures, such as analyzing and evaluating, newer models have come along.

There are at least three suitable replacements:

1. Revised Bloom's Taxonomy

In the mid-nineties, Bloom's taxonomy was updated to reflect a more active form of thinking and is perhaps more accurate (Anderson, Krathwohl, 2001):

This is perhaps the easiest replacement since it is closely related to the original taxonomy, thus most designers will rapidly adapt to it.. What is interesting about the updated version is how it resembles the SOLO Taxonomy (Structure of Observed Learning Outcomes):

2. SOLO Taxonomy

The SOLO taxonomy is a means of classifying learning outcomes in terms of their complexity in order to assess students' work in terms of quality (see http://edorigami.edublogs.org/2010/07/17/solo-taxonomy/)

3. Marzano's New Taxonomy

In The Need for a Revision of Bloom’s Taxonomy, Marzano describes six levels:

• Level 6: Self-system
• Level 5: Metacognitive System
• Level 4: Knowledge Utilization (Cognitive System)
• Level 3: Analysis (Cognitive System)
• Level 2: Comprehension (Cognitive System)
• Level 1: Retrieval (Cognitive System)

It is made up of three systems and the Knowledge Domain. The three systems are the Self-System, the Metacognitive System, and the Cognitive System. When faced with the option of starting a new task, the Self-System decides whether to continue the current behavior or engage in the new activity; the Metacognitive System sets goals and keeps track of how well they are being achieved; the Cognitive System processes all the necessary information, and the Knowledge Domain provides the content (see ftp://download.intel.com/education/Common/in/Resources/DEP/skills/marzano.pdf.).

What are your replacements for Bloom's Taxonomy?

Reference

Anderson, L.W., and Krathwohl, D.R., eds. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.

The Mosaic of Learning Styles

Yes I'm a few days late for David Kelly’s Learning Styles ‘Awareness’ Day, so I hope you forgive me. While most of the recent posts on using learning styles in instructional design have been mainly against using them, I'm going to take a slightly different position—not that we need to cater to each individual style, but that learning styles may be helpful when designing learning platforms.

So far the learning style debate has been mostly two tiles of a different color laid side by side—you are either fer it or agin it—we should assess student learning styles to improve learner outcome verses learning style assessments are unreliable, thus they should not never be used. However, I see the debate more as a mosaic that allows multiple patterns to occur.

Sensing and Intuitive Learning Styles

Perhaps the most critical study on learning styles is Coffield, Moseley, Hall, and Ecclestone's Learning styles and pedagogy in post-16 learning: A systematic and critical review. While the authors mostly found that matching the form of instruction to individual learning styles did not improve learning, there are some interesting exceptions throughout their paper, for example, on page 67 they write:

“More positively still, Katz (1990) in a quasi-experimental study of 44 occupational therapy students in the US and 50 in Israel, hypothesised that students whose learning styles matched the teaching method would perform better (ie more effectively) and would need less time to study outside class (ie more efficiently). The findings in both countries supported the premise that ‘the better the match is between students' individual characteristics and instructional components, the more effective or efficient the learning program is’ (Katz 1990, 233). But even this conclusion needed to be qualified as it applied only to higher-order cognitive outcomes and not to basic knowledge.”

So in search of a good paper on using learning styles in higher order cognitive skills I came across this paper, An Investigation into the Learning Styles and Self-Regulated Learning Strategies for Computer Science Students, by Alharbi, Paul, Henskens, and Hannaford. For their study they use the Felder-Silverman Learning Style model that uses four dimensions:

• Perception (Sensing or Intuitive) describes the ways in which learners tend to perceive information. Sensing learners prefer to learn facts, are comfortable with details, and tend to solve problems using well-established methods. Intuitive learners prefer abstract concepts, theories, and mathematical formulas, and seek innovation and new ideas when solving problems.
• Input (Visual or Verbal) distinguishes between learners based on their preferred medium for the presentation of information. Visual learners prefer to learn using visual medium of presentations, such as pictures, charts, and diagrams. Verbal learners prefer spoken or written materials. Both types of learners benefit when material is delivered using a combination of visual, verbal, and written forms.
• Processing (Active or Reflective) evaluates learners based on the way they process information. Active learners prefer to learn material by using it, whereas reflective learners prefer to think about how things work before actually trying them out. Active learners are typically more comfortable working in groups than reflective learners.
• Understanding (Sequential or Global) looks at how users understand new information. Sequential learners like to follow a step-by-step linear approach that focuses on the connections between the different parts of the learning material. Global learners prefer to grasp the full picture before narrowing into the details.

The study was not about assessing the learners' styles and then catering to their preferred styles but rather assessing them on the above dimensions and then testing them on a core computer science course to see how each dimension performed. The author's correlation analysis showed that while three of the dimensions (Input, Processing, and Understanding) were not statistically significant; the Perception dimension had a significant impact on the students' results in the examination, with the t-tests confirming that sensing students were significantly outperformed by intuitive students.

The authors note that the majority of students in the study (65.8%) were sensing learners, with 39.5% having a moderate or strong preference to that learning style. However, 21.0% of students have a moderate or strong preference to intuitive learning over sensing learning. This suggests that there is a need for learning material for both types of learners, but the greater emphasis should be placed on reducing abstraction to better meet the requirements of the sensing learners, especially when it is seen that intuitive learners performed significantly better on the midterm examination.

While it was just one study, it did seem to follow the patterns of a couple of studies discussed in the Coffield et al. paper:

• Woolhouse and Bayne (2000) noted that individual differences in the use of intuition are correlated with the sensing-intuitive dimension (p50)
• Allinson and Hayes (1996) report that intuitive students performed significantly better than analytic students on the Watson-Glaser Critical Thinking Appraisal (p85)

The authors write that instructors often tend to use more intuitive type instructions (abstract concepts, theories, etc.), rather than the more sensing types of instruction (such as facts, details, and problem-solving methods). While this might at first seem laudable in that they are trying to teach the learners to operate in a more complex world that seeks innovation and new ideas when solving problem, learners often need a basic scaffold of facts and basic problem-solving methods. Yes, some of the learning platforms that we might be providing are for complex environments that do not have proven problem-solving methods, but the least we should do is provide them with some simple facts and heuristics. For example, branching scenarios are often used in elearning platforms but we expect them to jump right in and guess their way through the activity.

An example of this is that one of the myths in our profession is that ISD or ADDIE was only designed for classrooms and there is a lack of rules for when classroom training should be used (we use it more often than it is needed), but the Armed Forces came up with a simple heuristic back in the 1980s - ADDIE Does More Than Classrooms. Thus this heuristic should be given to learners studying to be instructional/learning designers BEFORE they attempt to do a branching scenario or similar activity.

The Continuum of Learning Styles

In the Coffield et al. paper they note that the various theories of learning styles can be placed on a continuum (pp 9-10) as shown in the chart below. The ones on the left are considered more constitutionally fixed styles (innate) while the ones to the right are considered more flexible:

Click to bring up a larger chart in a new window

The Sensing and Intuitive learning styles discussed above fall on the right side of the continuum, thus depending upon the learner's knowledge and skills, the subject or task, and/or the type of instruction, a learner could fall on either the sensing or intuitive side of the dimension (however, from the studies noted in this post, the majority seem to fall on the sensing side).

One of the styles that fall strongly on the left side of the continuum is VAK (Visual, Auditory, and Kinesthetic), which poses a conundrum in learning styles.

The VAK Conundrum

In an interesting study, Visual Learners Convert Words to Pictures, functional magnetic resonance imaging (fMRI) technology was used to scan subjects' brains while they performed a novel psychological task involving pictures that could be easily named and words that could be easily imagined.  They found that the more strongly an individual identified with a visual cognitive style, the more they activated the visual cortex when reading words. Conversely, fMRI scans also showed that the more strongly an individual identified with a verbal cognitive style, the more activity they exhibited in a region of the brain associated with phonological cognition when faced with a picture.

Thus it seems our tendency to identify with being a visual or verbal learning is hardwired in us, however, visual preference does not always equal spatial aptitude (Ruth Clark & Chopeta Lyons, Graphics for Learning, 2004). Spatial aptitude is the ability to generate and retrain spatial images as well as transform images in ways that support visual reasoning.

Thus the conundrum—we may identify with being a visual or verbal learner (indeed, we may even be wired for one or the other), but it does not mean we are a good visual or verbal learner! Thus if we know what style our preference is, we need to think twice if we attempt to train others or learn something on our own if the learning method matches our style.

However, Clark and Lyons give us a few rules to follow:

1. Learners with low prior knowledge need graphics that are congruent with text (and preferably the text should be audio to prevent cognitive overload).

2. Learners with high prior knowledge need only words or visuals; not both, but one study did suggest that the diagram alone was best.

3. Encourage visual literacy. Some learners tend to view visuals as fluff, thus they tend to ignore them even though they might be their best means of learning. One method of encouraging their use is to use a visual and ask a question that can only be derived by examining the visual.

My Three Tiles in the Mosaic of Learning Styles

Sensing and Intuitive Learning Styles, The Continuum of Learning Styles, and The VAK Conundrum are my three tiles in the mosaic of learning styles. What are yours?