6.25.2014

Bloom's Revised Taxonomy: Cognitive processes and levels of knowledge matrix

Bloom's Revised Taxonomy (Remember - Understand - Apply - Analyze - Evaluate - Create) not only improved the usability of it (using action words), but perhaps also made it more accurate. However, probably the best feature — the matrix — is often left unnoticed. While Bloom's original cognitive taxonomy did mention three levels of knowledge or products that could be processed (shown below), they were not discussed very much and remained one-dimensional. The three levels are:

  • Factual - The basic elements students must know to be acquainted with a discipline or solve problems.
  • Conceptual – The interrelationships among the basic elements within a larger structure that enable them to function together.
  • Procedural - How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods.

In Krathwohl and Anderson's revised version, the authors combine the cognitive processes with the above three levels of knowledge to form a matrix. In addition they added another level of knowledge - metacognition:

  • Metacognitive – Knowledge of cognition in general, as well as awareness and knowledge of one’s own cognition. 

When the cognitive and knowledge dimensions are arranged in a matrix, as shown below, it makes a nice performance aid for creating performance objectives:

The Cognitive Dimension

The Knowledge Dimension Remember Understand Apply Analyze Evaluate Create
Factual            
Conceptual            
Procedural            
Metacognitive            

However, others have identified five contents or artifacts (Clark, Chopeta, 2004; Clark, Mayer, 2007):


  • Facts - Specific and unique data or instance.

  • Concepts - A class of items, words, or ideas that are known by a common name, includes multiple specific examples, shares common features. There are two types of concepts: concrete and abstract.

  • Processes - A flow of events or activities that describe how things work rather than how to do things. There are normally two types: business processes that describe work flows and technical processes that describe how things work in equipment or nature. They can be thought of as the big picture, of how something works.

  • Procedures - A series of step-by-step actions and decisions that result in the achievement of a task. There are two types of actions: linear and branched.

  • Principles - Guidelines, rules, and parameters that govern. It includes not only what should be done, but also what should not be done. Principles allow one to make predictions and draw implications. Given an effect, one can infer the cause of a phenomena. Principles are the basic building blocks of causal models or theoretical models (theories).

Thus the matrix might look similar to this:

The Cognitive Dimension

The Knowledge Dimension Remember Understand Apply Analyze Evaluate Create
Facts            
Concepts            
Processes            
Procedures            
Principles            
Metacognitive            

An example matrix that has been filled in might look something like this:

The Knowledge Dimension Remember Understand Apply Analyze Evaluate Create
Facts list paraphrase classify outline rank categorize
Concepts recall explains demonstrate contrast criticize modify
Processes outline estimate produce diagram defend design
Procedures reproduce give an example relate identify critique plan
Principles state converts solve differentiates conclude revise
Metacognitive proper use interpret discover infer predict actualize

For more on Bloom's Taxonomy, see:

5.05.2014

A to Z of Learning

aLearning (Alpha Learning) - To begin to understand, but not fully comprehend (has not groked or gLearned)

bLearning - (Blended Learning) - A mixture of media, such as cLearning and eLearning (brick and click)

cLearning - (Classroom Learning) - Learning in a classroom or formal setting

dLearning - (distance Learning) - Learning from an electronic device, such as eLearning, mLearning, or pLearning. May also include other media that is sent to the learner, such as mail

eLearning - (Electronic Learning) - Learning from the Internet (part of dLearning)

fLearning - (Formal Learning) - The learning goals are determined by instructional designers (learners may provide input)

gLearning - (Grok Learning) - To understand profoundly and intuitively (also see aLearning)

hLearning - (Horizontal Learning) - Learning something based on a horizontal continuum (there is no one right answer). It can best be learned in an collaborative learning environment (also see zLearning)

iLearning - (Informal Learning) - The learning goals are determined by the learner (organization may provide input)

jLearning - (Just in Time Learning) - Learning something at the moment of need and it sticks (not easily forgotten, also see qLearning)

kLearning - (Kindergarten Learning) -The basic skills Learned in Kindergarten that carry you throughout life

lLearning - (Laptop Learning) - Learning from a laptop computer

mLearning - (Mobile Learning) - Learning while away from home or office with a smart phone (may include pLearning or lLearning)

nLearning - (nonformal Learning) - The learning goals are determined by the business unit, such as manager or supervisor, normally used with oLearning (OJT)

oLearning - (OJT Learning) - Learning on the job through the direction of others (also see vLearning)

pLearning - (Pad Learning) - Learning from an electronic pad, such as an iPad or Galaxy

qLearning - (Quick Learning) - Learning something at the moment of need and it does not stick or is never used again (also see jLearning)

rLearning - (Redundant Learning) - Learning until one has thoroughly mastered a skill (trying to learn more will be useless). Sometimes call over-learning. Security and medical teams often use this method so it becomes second-nature

sLearning - (Social Learning) - Learning with or through other people (see uLearning for opposite)

tLearning - (Team Learning) - Used when two or more members of a team must learn different tasks that must be coordinated so that it increases the effectiveness of the team

uLearning - (Unilateral Learning) - Learning without the help of others (see sLearning for opposite)

vLearning - (Various Learning) - Learning various perspectives by gaining experience (normally on the job). Differs from oLearning in that vLearning is directed by the learner rather than a manager

wLearning - (Walking Around Learning) - Learning while walking around, normally through observation (adapted from Tom Peters' managing by walking around)

xLearning - (Xenodochial Learning) - Learning from others, especially in a diverse group of people who know little about each other, and they expect nothing in return

yLearning - (Yeoperson Learning) - Learning something that will promote the good of the common people or people in need

zLearning - (Zebra Learning) - Learning that is monochrome in nature (has only one correct answer or shade). May be easily learned through a cooperative learning environment (see hLearning for opposite)

3.16.2014

Simplicity Combats Complexity

James Murphy, a former F-15 fighter jet pilot and now CEO of a team building company, notes in a recent article that “complexity is the mortal enemy of good execution, and our world is nothing if not increasingly complex.” Thus, in order to execute in an ever increasing complex word, we need to break it down in simple steps:

  • Planning: If your planning process tries to run the whole gauntlet of complexity then you will simply get beat up by it, so use short-tem timelines.
  • Briefing; Get everyone on the same page.
  • Executing: Just do it.
  • Debriefing. The purpose of the debrief is to make adjustments ... to discover lessons learned. This can best be accomplished by performing an After Action Review:
    • What were our intended results?
    • What were our actual results?
    • What caused our results?
    • What will we sustain or improve?
  • Repeat process so that you are refining and continually improving.

OODA

Col. John Boyd, USAF (Ret) has a similar process called the OODA Loop:

  • Observe: Scan the environment and gather information from it.
  • Orient: Use the information to form a mental image of the circumstances and place it into context.
  • Decide: Consider options and select a subsequent course of action.
  • Act: Just do it.
  • Repeat process so that you are refining and continually improving.

Both Murphy and Boyd say that no matter how complex the environment is, when you do this, you stay at the same rate of competitive change in the complex environment or slightly ahead of it ... thus you win. Murphy also notes that confidence leads to courage, and courage leads to a bias towards action. This planning process gives people that courage regardless of the situation.

Probe, Sense, Respond

David Snowden of Cognitive Edge also has a similar tool for dealing with complexity in his Cynefin model:

  • Probe: Make a change (prototype) in the environment in order to test it.
  • Sense: Review it by determining the impact of the probe.
  • Respond: Depending upon the result you achieve you either amplify the probe or suppress it, and then repeat.

Agile Design

And the last one for dealing with complex environments — Agile Design:

  • Select the project and develop the vision.
  • Initiate the project by obtaining stakeholder participation, funding, and build team.
  • Deliver small working iterations that meet the changing needs of the stakeholders. Continue this step until:
    • Release (End Game) by deliver the final package.
    • Production: operate, maintain and support the system.

There are four other design models beside Agile that you can use depending on the complexity of the environment.

These tools, Murphy's Process, AAR, OODA, Cynefin, and Agile Design are designed for working in complex environments. What other processes or models do you use for dealing with complexity?

3.03.2014

Full Spectrum Learning

The U.S. Army has developed its answer to the 70-20-10 Learning model and Dan Pontefract's 3-33 Pervasive Learning model. However, they did a couple of twists by:
  • dumping the percentages
  • combining Experience with Social Learning
  • adding Education
  • adding two continuums - Responsibility and Ambiguity
Full Spectum Learning

Percentages

70-20-10 has been problematic in at least two ways. As Dan Pontefract notes in his book, Flat Army, it is based on leaders who were in charge of hierarchical, command and control cultures that were prevalent in the 1980s. While the U.S. Army does have a hierarchical command and control culture due to its nature, it is also composed of flat or horizontal teams (large and small) that operate alone and with each other in complicated environments that often border on the edge of chaos. Thus, it is both a hierarchical and flat organization that not only approximates how most successful organizations operate today, but is also based on all people, rather than just senior leaders.

Secondly, the use of percentages or ratios, such as 70-20-10 and 3-33, imply that they are predictive models, rather than reference models. In fact, the creators of 70-20-10 wrote that it is a predictive model. This can be noted in dozens of blog posts in which some very smart authors note 70-20-10 is a predictive model model and then are told in the comment sections that it is a reference model. If you do a image search on Google for the term "reference model" (may NSFW as it shows a couple of nude models) you will notice that none of the images are based on percentages or ratios.

Experience has Social Learning in a Learning Environment

The Full Spectrum Learning model realizes that if you are gaining experience to learn, then it is implied that you will be using plenty of informal and social learning, along with smaller amounts of training and education. In order to build skills and knowledge via experiences, the environment must contribute to peer-based learning through blogs, wikis, micro-blogs, and other social based media. It leverage these social tools to build dynamic vertical and horizontal social networks for formal and informal information sharing in order to foster critical thinking and problem solving skills needed for operational adaptability.

The Addition of Education

In Human Resource Development, training is normally associated with learning to perform a present job or task, while education is normally associated with learning to perform a future job or task. Thus, in a rapidly changing world, education through formal and/or nonformal environments is a required component if an organization wants to remain competitive. For example, during our last recession, companies dumped thousands of people in mass layoffs. Now they are whining that they cannot find people who have the education and training that they require. Good organizations should always be building a path towards the future by educating people to walk that path.

The Responsibility and Ambiguity Continuums

Rather than build a learning model that focuses on one fixed point, the U.S. Army created the Full Spectrum Learning model on two continuums based on the degree of responsibility of the learner and the degree of ambiguity of the learning environment to give it depth.
What are your thoughts on the three models?

10.31.2013

I wrote a new chapter in my leadership manual - Path-Goal Theory

Path-Goal Theory of Leadership

6.04.2013

Designing in the Linear... but Repeated

Prophet. "You value your ignorance of what is to come?"

Sisko, "That may be the most important thing to understand about humans. It is the unknown that defines our existence. We are constantly searching, not just for answers to our questions, but for new questions. We are explorers. We explore our lives, day by day. And we explore the galaxy, trying to expand the boundaries of our knowledge. And that is why I am here. Not to conquer you with weapons or ideas, but to co-exist and learn."

- Deep Space Nine: Emissary

Harvest Moon over Lynnwood

Sometimes we think of linear design as strictly a step-by-step process that holds no possibility of using our imagination. However, the reality is that when we design in the linear an unlimited number of possibilities can occur. We generally try to set some sort of strategy with our design. With each new consequence that our strategy creates, the final design begins to take on more shape. But we never really know what will happen until we come to the end.


In contrast, dynamic design (nonlinear) is simply the repetition of a series of linear. This is because we are locked in by the concept of time — we exist in the linear. Of course, each repetition should bring something new or we will simply be repeating ourselves. The value of repeating the linear by bringing something new to it is that it brings the possibility of learning from new information. That is, our previous linear experience brought about certain consequences... some of them unintended, thus we now have new information that we can connect with our design.


Flower
“The value of any piece of information is only known when you can connect it with something else that arrives at a future point in time,” - Ira “Gus” Hunt.
The value of dynamic design is that it allows us to connect new information that we previous didn't have. Each repetition of the linear is called an iteration that normally comes in two forms:

  • Design or interpretive iteration — testing a learning method, function, feature, etc. of the learning or training process to see if it's valid.
  • Release iteration — releasing it to the business unit or customer even though it may not be fully completed or functional because we believe that it is good enough to be of use.
We have at least five design methodologies to use, depending upon our needs (see A Table of Five Design Models):

ADDIE or ISD

ADDIE normally uses two types of Procedural Analysis Methods. When the task to be learned is primarily overt behavior, there are set procedures to follow, and you have an expert performer, Behavioral Task Analysis is normally used. Since this is one of the more easier analysis to perform, few or no design iterations are required.

The Information Processing Analysis tool is used when there are both overt steps that require a set order and covert steps that require decision making of a yes or no nature (if, then, else), which means Flow Charts are good tools for this method. Since there are behaviors that cannot be seen, more design iterations are normally needed.

Design Thinking

Design Thinking often uses Rule Based Analysis Methods as there are no set procedure for performing the task and most of the task steps are normally of an overt nature.

Two forms of analysis can be used, GOMS Analysis and the Critical Decision Method. The task to be learned may best be represented by one or the other and sometimes both.

GOMS analysis is good if you can readily determine the Goal, Operations, Methods, and Selection rules (see Rule Based Analysis Methods). Since you have four main branches, mind or concept maps are good tools to use. Because the behaviors are mostly covert, several design iterations are normally used in order to fully capture all the required behaviors.

If you have an expert performer who has recent experience then the Critical Decision Method is a good method as it allows you to capture their stories in a case study format.

Agile Design

Agile Design is primarily used when the final product (learning or performance process) will take some time to fully complete but can be of use to the customers. Thus, release iterations are used as the benefit is that the customers will get part of the product without having to wait for long periods of time.

Both Rule Based Analysis and Procedural Analysis Methods are normally used, however the problem should be complex enough that it will take several weeks or months to complete the final product (if it only takes a short while there is little need to make constant release iterations).

System Thinking

The System Thinking method is for processes that will be implemented across several parts of the organization. Since each part normally has particular needs that must be met it often requires that trial tests (iterations) be run in each part to ensure it meets all the customers needs. The result of these iterations will determine the need, if any, for more iterations while the product is fined tuned for each part of the organization.

X Problems

We are constantly searching, not just for answers to our questions, but for new questions. It is this unknown that defines our existence for it is at this point that we can expand the limits of our knowledge. The X Problems method is a good choice for this exploratory nature in us. Since it is exploratory, several design iterations are often required. In addition, release iterations are also often used if it is believed that what we learned so far will be of use to our customers.


5.07.2013

If all you have is a hammer. . .

Hammer

Then everything looks like a nice craft beer. . . or a nail:
Hammer and bottle opener

Nowadays we expect so much more from our tools:
Multi-tool Hammer

We expect them to be multi-functional. . . to include ADDIE and the Four Levels of Evaluations.

5.01.2013

MERLOT 2013 Classics Award

I'm honored to announce that Big Dog's Leadership Page won a MERLOT 2013 Classics Award for developing an outstanding, peer reviewed online resource.

MERLOT Award

It is recognized as an exemplary model for business.

4.29.2013

Flat Army: 3-33 Learning Model Verses 70-20-10

I have been reading Dan Pontefract's new book Flat Army: Creating a Connected and Engaged Organization, and can honestly say it is an excellent and insightful book for developing a collaborative and open leadership organization. The part that grabbed my attention the most is the section in chapter 9, Learning at the speed of need, which discusses the 70-20-10 learning model (while I have a great interest in all of Dan's topics in his his book, learning is my primary field of interest).

Dan notes that there is no empirical evidence that learning maps to the 70-20-10 model, even though practitioners often cite it as a fact. It was developed in the 1980s when command and control was at the heart of leadership—think of Lee Iacocca, Jack Welch, and Roger Smith who more than likely thought that Learning/Training Departments were solely for their bidding and could offer very little for them personally. Thus the model is based on the the very thing that many learning and development practitioners are trying to get away from—hierarchy organizations. In addition, the model was developed before the Internet, thus it does not account for the numerous technologies that have aided formal learning, such as just-in-time learning, elearning, virtual learning.

I would also add that while some have pointed to Where did the 80% come from? as further proof that the 70-20-10 learning model is valid, when I researched the listed references on the page that give low percentages to formal learning, such as Raybould who proclaims that formal learning only accounts for 10% of the learning, I discovered that the authors provided no evidence at all. Such citations seam to imply, “I saw it on the Internet so it most be true.” On the other hand, the references that provide higher ratios for formal learning are the most evidence based.

The 3-33 Model

Dan provides what I see is a closer approximation of the learning ratios: 3-33, which stands for 33% the learning is formal, 33% is informal, and 33% is social. What is most interesting is that the research behind his model revealed that when the learners were asked to give the percentages on how they thought they learned, the numbers were very different than when the researchers actually discovered how the learners did indeed learn. This coincides with other research that indicates what learners are able to judge about their learning experiences (see Learner Self-Assessment Ratings).
3-33 Learning Model
graphic by Dan Pontefract

One of the other major errors of the 70-20-10 model is that it places reading in formal learning. Since when did reading a book become formal learning? Dan of course places it under the correct type of learning in his 3-33 model... informal. The 70-20-10 error seems to again coincide with the command and control culture that was most prevalent in the 1980s—the top leaders viewed writers as part of the elite who they could trust and learn from, while the learning/training functions were viewed as something to command and control, rather than trusted partners.

All in all, Flat Army is a very good book that deserves a special spot in in anyone's library who is interested in collaboration, leadership, and learning.

DISCLAIMER: While Dan and I have exchanged comments via blogs, I have no other interests in the book.

NOTE: While I seem to be kind of harsh on “command and control” it is mostly because of the audience I write for. I'm retired military so my real view of command and control follows the military view, which vastly differs from layman's and others point of view. See my writings on Leadership, Management, Command, & Control.

What are your thoughts on the two models?