Big Dog, Little Dog

Kirkpatrick's Revised Four Level Evaluation Model

2012-01-16T19:33:00.000-08:00

I had an interesting discussion with Clark Quinn on using Kirkpatrick's model in learning processes other than courses. Clark argues that use of Kirkpatrick’s model is only for courses because training is the dominant discussion on their web site. I disagree and wonder if perhaps it is more of a “not invented here” hesitation because advancing concepts to the next level has often been a primary means of moving forward. It might sound good to forget an old model, but if you do not help people relearn, then their old concepts have a nasty habit of reappearing. In addition, training is far more than just courses. So after some heavy reflection I did a rewrite on my Kirkpatrick web page and have listed some of the highlights below.

More than Courses

While some mistakenly assume the four levels are only for training processes, the model can be used for other learning processes. For example, the Human Resource Development (HRD) profession is concerned with not only helping to develop formal learning, such as training, but other forms, such as informal learning, development, and education (Nadler, 1984). Their handbook, edited by one of the founders of HRD, Leonard Nadler, uses Kirkpatrick's four levels as one of their main evaluation models.

Kirkpatrick himself wrote, “These objectives [referring to his article] will be related to in-house classroom programs, one of the most common forms of training. Many of the principles and procedures applies to all kinds of training activities, such as performance review, participation in outside programs, programmed instruction, and the reading of selected books” (Craig, 1996, p294).

Kirkpatrick's levels work across various learning processes because they hit the four primary points in the learning/performance process... but he did get a few things wrong:

1. Motivation, Not Reaction

Reaction is not a good measurement as studies have shown. For example, a study shows a Century 21 trainer with some of the lowest reaction scores was responsible for the highest performance outcomes in post-training (Results) as measured by his graduates' productivity. This is not just an isolated incident—in study after study the evidence shows very little correlation between Reaction evaluations and how well people actually perform when they return to their job (Boehle, 2006).

When a learner goes through a learning process, such as an elearning course, informal learning episode, or using a job performance aid, the learner has to make a decision as to whether he or she will pay attention to it. If the goal or task is judged as important and doable, then the learner is normally motivated to engage in it (Markus, Ruvolo, 1990). However, if the task is presented as low-relevance or there is a low probability of success, then a negative effect is generated and motivation for task engagement is low. Thus it is more about motivation rather than reaction.

2. Performance, Not Behavior

As Gilbert noted, performance has two aspects: behavior being the means and its consequence being the end... and it is the consequence we are mostly concerned with.

3. Flipping it into a Better Model

The four levels are upside down as it places the two most important items last—results, and behavior, which basically imprints the importance of order in most people's head. Thus by flipping it upside down and adding the above two changes we get:

Result - What impact (outcome or result) will improve our business?
Performance - What do the employees have to perform in order to create the desired impact?
Learning - What knowledge, skills, and resources do they need in order to perform? (courses or classrooms are the LAST answer, see Selecting the Instructional Setting)
Motivation - What do they need to perceive in order to learn and perform? (Do they see a need for the desired performance?)

With a few further adjustments, it becomes both a planning and evaluation tool that can be used as a troubling-shooting heuristic (Chyung, 2008):

The revised model can now be used for planning (left column) and evaluation (right column).

In addition, it aids the troubling-shooting process. For example, if you know the performers learned their skills but do not use them in the work environment, then the two more likely troublesome areas become apparent as they are normally in the cell itself (in this example, the Performance cell) or the cell to the left of it:

There is a process in the environment that constrains the performers from using their new skills, or
the initial premise that the new skills would bring about the desired change is wrong.

The diagram below shows how the evaluation processes fit together:

Learning and Work Environment

As the diagram shows, the Results evaluation is of the most interest to the business leaders, while the other three evaluations (performance, learning, and motivation) are essential to the learning designers for planning, evaluating, and trouble-shooting various learning processes; of course the Results evaluation is also important to them as it gives them a goal for improving the business. For more information see Formative and Summative Evaluations.

I go into more detail on my web page on Kirkpatrick is you would like more information or full references.

What are your thoughts?

Visualization (Sensemaking) in Rapid Agile Learning Design

2012-01-09T13:59:00.000-08:00

Common definitions of visualization usually read something like, “to form a mental image,” thus we often think of visualization as being a simple solo technique, such as picturing “a dog eating a bone” or “a person doing the right thing.” However in an organization context, visualization is much more complex in that while it involves an image of the working environment, it is also a complex process that is very social in nature.

The Visualization Framework

Visualization is often used interchangeably with sensemaking—making sense of the world we live and operate in, and then acting within that framework of understanding to achieve desired goals. Thus visualization is not just a shared (social) image with intent, it also implies ACTION. This framework can be used for building agile or rapid learning designs, fixing performance problems, implementing informal learning solutions, etc.

Visualization Framework

Visualization Framework (opens larger image in a new window)

The start of a visualization process is often sparked by a cue from the environment, such as an increase in customer complaints; or a team charged with improving a process. The steps within the visualization or sensemaking framework include (Leedom, McElroy, Shadrick, Lickteig, Pokorny, Haynes, Bell, 2007):

1. Triggering cues (information that acts as a signal) from the environment are perceived by the people in a Community of Interest (CoI). These cues may be picked up by one or more members of the CoI. A couple of examples of triggering cues might be an increase in the number of customer complaints or an unexpected drop in production.

2.Triggering cues create a situational anomaly—facts that do not fit into the framework of familiar mental models. Detection of these anomalies violate the expectancies of the members of the CoI and creates a need for change (improvement).

Note: A mental model is a structure or frame that is built from past experience and becomes part of an individual’s store of tacit knowledge. It is comprised of feature slots that can be instantiated by information describing a current situation (such as triggering cues). Its functional purpose allows a person to assess the situation, take a course of action, follow causal pathways, and recognize constraints in order to achieve a set of goals for actively confronting the situation. Fragmentary mental models can often be linked together to form a just-in-time explanation of a situation. Examples of a mental model include a chess player reacting to a move on the chessboard, a doctor diagnosing a medical condition, or an instructional designer solving a performance problem.

3. Specific data from the information environments trigger the mental activation of familiar mental models. The members of the CoI analyze and discuss the anomalies until they discover a purposeful structure or pattern for interpreting the new information. This transforms the problem space into various solutions. This process of “pattern matching” starts the basis for constructing new or revised mental models. Since patterns differ among the members, they collaborate by telling stories, metaphors, etc. to build common understanding.

4. Activation of a specific mental model is typically triggered by matching salient facts to one or two key features that uniquely anchor a new model that the CoI can agree upon. Tacit knowledge or intuition is often used to build mental models and the degree of tacit knowledge will vary among the members, thus they use a “negotiation process” to ensure all needs are met (or at least prioritize them according to available resources).

5. An action plan is used to instill the selected mental model into the work space in order to transform it to the desired state (during the visualization process intent must always be associated with action, otherwise it is just wistful thinking). The action plan includes the final development of any needed content, material, or products. Once all the pieces are put together, the action plan is implemented.

6. New information from the transformation process is perceived by the CoI, which in turn processes it to determine if the patterns match their desired mental model.

7. If the new information does not match the CoI's newly constructed mental model (situational anomalies are again perceived and they may or may not differ from the original ones), then the visualization process begins anew.

Probing and Shaping

While the visualization process does use passive information that derives from experience and expertise, it also involves the proactive use of shaping actions to reduce risk and uncertainty and probing actions to discover system effect opportunities that can then be exploited.

Probing develops greater understanding by experimentally testing the operational environment, such as asking questions, Cognitive Task Analysis, or immersing oneself in the troubled environment to discover new information. These probing actions help to illuminate key structures and linkages within the environment.

Shaping is taking an incentive action to discover new information in order to determine if it aids in transforming the troubled environment to meet the new mental model. Prototyping may be used as a shaping tool—an iterative process of implementing successive small-scale tests in order to permit continual design refinements. There are normally two types of prototypes:

Design Iteration (interpretive) — the iteration is performed to test a learning method, function, feature, etc. of the action plan to determine if it is valid.
Release Iteration (statistical) — the iteration is released as a product to the business unit or customer. Although it may not be fully completed or functional, the designers believe that it is good enough to be of use.

Probing actions serve to illuminate additional elements and linkages within the visualization space that can then be subsequently exploited for operational advantage.

Visualization is Dynamic, Not Static

The visualization or sensemaking framework in not linear, but rather a dynamic process that may flow in any direction, for example:

The Dynamics of Visualization

Dynamics of the Visualization Framework (opens larger image in a new window)

A Community of Interest holds a vested interests when faced with a troubling situation, thus they need a dynamic model that aids them in fulfilling their mission within complex environments. The military has a term called “center of gravity,” which is defined as the source of power that provides moral or physical strength, freedom of action, or the will to act. It is seen as the source of strength of the organization. The ability to act upon and transform an under-performing environment through the use of visualization or sensemaking is an essential attribute in an rapidly moving environment in that it helps to ensure the center of gravity stays balanced.

Reference

Leedom, D. K., McElroy, W., Shadrick, S. B., Lickteig, C., Pokorny, R. A., Haynes, J. A., Bell, J. (2007). Cognitive Task Analysis of the Battalion Level Visualization Process. Arlington, VA: United States Army Research Institute for the Behavioral and Social Sciences. Technical Report 1213. Retrieved on January 5, 2012 from http://www.hqda.army.mil/ari/pdf/TR1213.pdf

Learning Styles are for the individual, not group

2011-12-07T15:33:00.000-08:00

NOTE: I left this comment in eLearn Magazine's, Why Is the Research on Learning Styles Still Being Dismissed by Some Learning Leaders and Practitioners by Guy Wallace. Since it wiped out most of my formatting, such as comments and quotation marks, I am posting it here for better readability.

Perhaps one of the best papers on learning styles is Coffield, Moseley, Hall, and Ecclestone's, Learning styles and pedagogy in post-16 learning: A systematic and critical review (PDF). While the paper does dismiss some types of learning styles and the importance that the recognized learning styles actually have when it comes to learning, it does leave a lot of questions opened.

One of the most profound statements in the paper, at least to me, is (p68):

“just varying delivery style may not be enough and... the unit of analysis must be the individual rather than the group.”

That is, when you analyze a group, the findings often suggest that learning styles are relative unimportant, however, when you look at an individual, then the learning style often distinguishes itself as a key component of being able to learn or not. Thus those who actually deliver the learning process, such as teachers, instructors, or trainers and are responsible for helping others to learn see these styles and must adjust for them, while those who design for groups or study them see the learning styles as relative unimportant.

In the next paragraph, the paper continues with this statement:

“For each research study supporting the principle of matching instructional style and learning style, there is a study rejecting the matching hypothesis’ (2002, 411). Indeed, they found eight studies supporting and eight studies rejecting the 'matching' hypothesis, which is based on the assumption that learning styles, if not a fixed characteristic of the person, are at least relatively stable over time. Kolb's views at least are clear: rather than confining learners to their preferred style, he advocates stretching their learning capabilities in other learning modes.”

While many find this as a reason to dismiss learning styles, I find it quite intriguing in that why do learning styles play a key component is some situations or environments, but not others? I think part of the answer is within this finding—a study that was conducted in the U. S. and Israel, found that when students' learning styles matched the teaching method they performed both more effectively and efficiently. But the authors of the paper seem too readily to dismiss it as the end the paragraph with this statement—“But even this conclusion needed to be qualified as it applied only to higher-order cognitive outcomes and not to basic knowledge.” (p67)

It seems logical that higher-order cognitive outcomes need more individual support (in this case matching the learning style the the correct learning strategy) than basic knowledge. Thus in some situations learning styles are important, while in others they are not.

Finally, in the paper's conclusion the authors note (P132-133) that:

“Despite reservations about their model and questionnaire (see Section 6.2), we recognise that Honey and Mumford have been prolific in showing how individuals can be helped to play to their strengths or to develop as all-round learners (or both) by means, for example, of keeping a learning log or of devising personal development plans; they also show how managers can help their staff to learn more effectively.”

Thus the main take-away that I get from the paper if that if you are an instructor, manager, etc. who has to help the individual learners, then learning styles make sense. On the other hand, if you are an instructional designer or someone who directs her or his efforts at the group, then learning styles are probably not that important. Note that I am both a trainer and a designer so perhaps this is why my take-away makes sense to me.

Lingering Doubts About the 70:20:10 Model

2011-11-29T11:08:00.000-08:00

In a couple of recent posts both Ben Betts and Clive Shepherd casts their doubts about the usefulness of 70-20-10 model and wonder if it's confusing the issue. You can read their posts at The Ubiquity of Informal Learning, and Beware who's selling informal learning.

I tend to agree with them, but before I begin I want to add that if you think I'm anti-informal learning, then please note that I wrote a post defending informal learning and it was Tweeted quite heavily. In addition, I've seen in the comments of these posts and others that if you challenge the idea about the usefulness of the 70-20-10 model then either you don't want to understand it, you clearly don't get it, or you see it as a threat to your job. If this is what you really think then you may talk-the-talk of informal and social learning learning but you walk-the-walk of a lecturer—“it's my way or the highway.” I have no patience with these attitudes because they are simply attacking people rather than their ideas.

While some proponents of the model insist it is non-prescriptive, both Ben Betts and Clive Shepherd saw the model as being “prescriptive.” I saw it as being prescriptive. Jay Cross saw it the same way as he wrote in one of his posts, A model of workplace learning,“The 70-20-10 model is more prescriptive. It builds upon how people internalize and apply what they learn based on how they acquire the knowledge.”

Even the Center for Creative Leadership, where the model was developed, write that the 70-20-10 model is indeed prescriptive:

“A research-based, time-tested guideline for developing managers says that you need to have three types of experience, using a 70-20-10 ratio: challenging assignments (70 percent), developmental relationships (20 percent) and coursework and training (10 percent).”

The 70-20-10 model is a prescriptive remedy for developing managers to senior and executive positions. Parts or perhaps all of the model may be useful for developing other professionals. However, by no means is it a useful model for the daily learning and work flows that takes place within organizations because it is being applied in an entirely different context that what it was designed to do. When people see numbers applied to a model they normally assume a couple of things: 1) that it is fact based, and/or 2) this is the way it is supposed to be.

As Will Thalheimer noted in one of his posts, adding numbers to make a model look more authentic makes it both bogus and dangerous (see People remember 10%, 20%...Oh Really?). I can attest to that because in some of my posts in the past I wrote that the formal to informal ratio was 30/70. People immediately commented and insisted it was 20/80 or 10/90. They seemed determined to lock the numbers in to an exact ratio—NO EXCEPTIONS! Even the model is begriming to look more like real ratios that must be adhered to because it is now being written as 70:20:10. Where will it end?

For more on the ratios see, 70-20-10: Is it a Viable Learning Model?

Yes, you can manage informal learning

2011-11-01T10:46:00.000-07:00

Jane Hart recently posted a thought-proving article on her blog in which she argues that you “can't manage informal learning, you can only manage the social media tools.” In her post she goes to great depth to define some of the various types of learning, such as formal, non-formal, and informal, however, I think the same needs to be done for “manage” in order to get a more accurate picture, otherwise we get mental images of Dilbert's pointy-haired boss when someone speaks or writes about management.

People often equate the term “management” with “control,” that is, when you manage something, you are trying to take direct control of it. However, management and control are actually two of four distinct processes for guiding an organization. The other two are leadership and command. While these are separate processes, they need to be blended together to deal with our rapidly changing world. Note that while I define the terms based upon my military experience and training, civilian organizations often use them because the military has the resources to study and research these concepts (and their studies are often done on civilian organizations which makes them valuable to the outside world).

Command and Control

Command is the imparting of a vision to the organization. It does this by formulating a well-thought out vision and then clearly communicating it. It emphasizes success and reward. That is, the organization has to be successful to survive and in turn reward its members (both intrinsically and extrinsically).

An example in this case would be visioning a process that helps to increase informal learning and make it more effective. A bad vision would be implementing a social media tool, such as a wiki or Twitter. This is because they are tools or technologies that are means rather than an end-goal.

Visions do not have to come from the top, but rather anywhere in the organization. Informal leaders are often good sources of visions, however if the vision requires resources, then they normally need the support of a formal leader.

In contrast, control is the process used to establish and provide structure in order to deal with uncertainties. Visions normally produce change, which in turn produces tension.

For example, “is the tool we provided to increase the effectiveness of informal learning really working?” Thus it tries to measure and evaluate. Inherent in evaluation is efficiency—it tries to make the goal more efficient. This can be good because it can save money and often improve a tool or process. The danger of this is if the command process is weak and the control process is strong then it can make efficiency the end-goal. That is, it replaces effectiveness with efficiency.

A good example of this is our present economy that caused many organizations to perform massive layoffs. Now the same organizations are complaining that they can't find qualified workers. Efficiency over road effectiveness—they failed to realize that they would need a trained workforce in the future.

Leadership and Management

Management's primary focus is on the conceptual side of the business, such as planning, organizing, and budgeting. It does the leg work to make visions reality. Thus it helps to acquire, integrate, and allocate resources to accomplish goals and task. This is why you need to manage non-formal learning and not just the tool itself. The goal is to increase informal learning and make it more effective, not to put into place a media tool. If the tool because the goal, then the wrong polices could be put into place that decrease its value as an informal learning tool.

Secondly, if the focus is only on the tool, then other options are omitted, such as tearing down cubicles and creating spaces where people can meet.

In contrast, leadership deals with the interpersonal relations such as being a teacher and coach, instilling organizational spirit to win, and serving the organization and workers.

Thus all four processes have their place. When you manage informal learning, you are not trying to control it, but rather planning how you will put the vision in place, budgeting for the required resources, and then organizing the teams so they can make it a reality.

In the August 2010 (p.10) edition of Chief Learning Officer magazine, Michael Echols notes a survey that the number one priority of 96 percent of the CEOs they surveyed want proof that learning programs are driving their top five business measures, but only 8 percent are getting it. Thus the learning and development leaders are going to start feeling the heat to get some type of evaluation process into place. If informal learning is going to be one of the primary objectives, we are going to have to get real about actually trying to measure it. The excuse that the learners control it so it can't be done is not going to cut it for long.

Mapping the Performance

2011-10-26T17:10:00.000-07:00

Part of the analysis phase is capturing the skills required for performance. While there are several methods for capturing these performance, ISD normally only lists Behavioral Task Analysis. However, many task are largely overt and nonprocedural in nature, thus they require a Cognitive Task Analysis. The solution of course is to simply plug the desired method or tool into the ISD or ADDIE model as it is quite dynamic rather than the stale linear model that some believe.

Four analysis tools will be discussed in this post that may be plugged into ISD:

Behavioral Task Analysis
Information Processing Analysis
GOMS Analysis
Critical Decision Method

Considerations for Analysis Tool Selection

Selecting the correct analysis tool is dependent on the type of actions the worker must perform. This performance is normally composed of two types of actions:

Overt - behavioral and observable
Covert - mental and not observable

While some some tasks are only composed of one or the other, more complex tasks may be composed of both actions.

In addition, the selection of the analysis tool is also dependent upon the task steps:

Procedural - the steps are performed in order and are normally largely overt actions
Rule Based - the steps do not have to be performed in a temporal order and are normally largely covert (cognitive) actions

Procedural Analysis Methods

These methods are used when there is a temporal order of involved steps, thus there is a set procedure for performing the task. Two analysis tools that fall under Procedural Analysis are Behavioral Task Analysis and Information Processing Analysis.

Behavioral Task Analysis

Behavioral Task Analysis is used to capture overt actions by observing and recording an Exemplary Practitioner perform the task. Questions may also asked to ensure the analyst has fully captured the performance. This is perhaps the easiest method. The output is a list of steps that may also have diagrams or pictures of the desired performance or behavior. A short example is:

Turn on computer and start spreadsheet application.
Load projected sales report spreadsheet template (prosale.exl).
Enter projected sales figures into designated spreadsheet cells.
Run spreadsheet macros.
Save file under new name, “pro***.exl”, with *** being the next sequential number For example, pro135.exl. Note: Do NOT overwrite template.
Forward to Planning Manager by email.
Exit application.

Depending upon the learners' prior knowledge and the complexity of the task, the list might also contain substeps as shown in this example:

Turn on computer
Start spreadsheet application

2.1 Click the Start icon
2.2 Scroll through the menu list and select the Excel application

Load projected sales report spreadsheet template (prosale.exl).
- 3.1 Click on the file menu
- 3.2 Click on the Open option
- Etc., Etc.

Information Processing Analysis

This tool is used when there are both overt steps that require a set order and covert steps that require decision making of a yes or no nature. The overt actions are captured by observing and recording an expert performer, while the covert actions are captured by having the expert performer talk about their actions (thinking aloud). Capturing the decision making process makes it more difficult than the Behavioral Task Analysis, but it is perhaps the most common method as the majority of tasks require decisions. The output is commonly a flow chart composed of three elements that outline the task steps:

Boxes - overt or covert actions
Diamonds - decisions
Arrows - order of steps

A short example of a flowchart showing a decision a forklift operator must make when moving goods from the receiving dock to a storage area may look like this:

Rule Based Analysis Methods

These methods are used when there is NO temporal order of involved steps, thus there is not a set procedure for performing the task. In addition, most of the task steps are normally of a overt nature. Two tools that fall under Rule Based Analysis are GOMS Analysis and the Critical Decision Method.

GOMS Analysis

The GOMS tool analyzes a task by examining four elements of it:

Goals represents the intention to perform a task its underlying structures, such as subtasks, cognitive operations, and physical operations. For example, an Instructional Designer may have a task of selecting activities that will give a new sales person the skills to sell a product. That task includes subtasks of analyzing the skills needed, analyzing activities, etc. The ID will use cognitive operations, such as selecting an activity that actually teaches the skill and determining if the chosen media can effectively deliver the activity.
Operations represent the physical actions, such as binding a learner's manual or pressing a button. It also includes mental operations, such as retrieving from memory or setting a goal.
Methods represent sequences of operations that accomplish goals or objectives. It includes high-level goals that breaks a tasks into subtasks and low-level methods that are the actions that actually perform the subtasks.
Selections Rules represent the context for selecting a particular method. That is, there may be several ways to accomplish a goal or task, but one may be chosen because it should perform this particular task the best (heuristics).

Capturing the performance of a covert task can be done in many ways and normally several are used to capture the task. Some of the more common ones are interviews, job shadowing, having an expert performer think-aloud, and storytelling.

The GOMS analysis can normally best be represented by a concept or mind map. The Goals will normally be placed on the map first, which is then followed by placing the Methods and their Selection Rules on the map. The details (Operations) are then listed. Shown below is a partial mind map of selecting an analysis tool:

Depending upon the size and the scope of the task, you might have to link or reference other documents that go into more specifics about the Operations, Methods, and Selections rules. However, be careful as one of the common mistakes in GOMS analysis is getting too specific, which results in long and detailed procedural descriptions that are difficult to follow.

Critical Decision Method (CDM)

This can be thought of as a Case Study, however, it also includes a visual reference or map. Just as a case study uses an actual incident to tell a story, CDM is also performed by having an expert tell a story about a particular task they performed in the past. For example, an Instructional Designer might tell how he developed a Just-in-Time program for training sales persons to sell a new product or a fire fighter might tell about the actions she took in fighting a gas station fire. Note that when the person tells the story, the interviewer might have to probe to gather some details. While the output might include a case study, it should normally include a map similar to this:

The CDM process goes like this:

Sweep 1 - Identify a incident. It should come from a decision maker who was involved rather than a witness.
Sweep 2 - The expert tells his or her story. Identify the key decision points and when they were made.
Sweep 3 - These is where you “deepen” the interview by asking for analogs, mental models, other options, experience or training that was helpful, etc.
Sweep 4 - Finally ask “What If” questions, such as “If the situation had been different what would have happened?”, or “What if a novice had been in charge?”

Key Points

The finale outputs, such as a list of task steps, flowchart, mind map, or CDM chart prove not only invaluable to the the design team, but may also make excellent performance or learning aids. Just as it is as important to use a correct analysis tool, it is also just as import to represent the findings in a manner that others can understand.

The final output might not be just one chart, but rather a combination of them as shown in the picture at the top of this post. Also the chart does not have to be combined into one page if it gets too cluttered, but may rather be composed of a collection of documents that are linked together electronically or referenced if they are printed. In addition, other types of charts and visualizations representations may also be used.

More detailed information can be found in two books, van Merriënboer's, Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training (1997) and Crandall, Klein, & Hoffman's, Working Minds: A Practitioner's Guide to Cognitive Task Analysis (2006). Merriënboer discusses Behavioral Task Analysis, Information Processing Analysis, and GOMS Analysis (plus other topics related to complex tasks), while Crandall, Klein, & Hoffman discusses Concept Mapping and Critical Decision Method (plus other topics related to Cognitive Task Analysis).

Both Merriënboer and Crandall, Klein, & Hoffman's books are excellent references and provide different information about Cognitive Task Analysis, however, Merriënboer's is now out of print, thus the used books may be a bit on the pricey side.

ADDIE Does More Than Classrooms

2011-10-17T07:33:00.001-07:00

One of the misconceptions of ISD is that it was created to only build classroom training environments.Yet, one of the old Army manuals (1984) that is used for training ISD shows the above options for training. It also notes that options 1, 2, or 3 should be used in lieu of classroom training if it can adequately perform the job. Thus, classroom training should normally be the last option if there is more than one viable option. This is because classroom training is normally one of the more expensive options.

The five options are shown below with a few notes about them:

Job Performance Aid (JPA) - this would include today's EPSS (Electronic Performance Support System)
Self-Teaching Exportable Package - the elearning we know today would fall under this category
Formal On-the-Job-Training (OJT)
Installation Support School (on or near the employees' workplace) - this would be formal classroom training even though the training may be conducted outside in the field
Resident Instruction (away from employees' workplace where travel and living expense would have to be considered) -this would also be formal classroom training.

Just as training designed for the classroom, the other options also need to follow the five phases of ISD to ensure they do what the are supposed to do. For example, even a simple JPA requires:

Analyzing the various settings and media to determine if it is the most approximate method.
Designing it so it performs as intended.
Developing it into a real product.
Delivering (implementing) it to the workers who need it.
Evaluating it to ensure it does the job it was intended to do. This also shows the business units that you care about the solutions you deliver (if it ain't worth following up on then it probably ain't worth doing) and you might learn something. Note the evaluation may be as simple as checking with a couple of managers and some of the employees to ensure it is doing what it is supposed to do.

The manual also gives some guidelines for selecting the correct training setting:

Job Performance Aid
- close supervision not required
- task follows a set procedure
- JPA can be followed while performing the task
- do not use if:
  - consequence of inadequate performance is high
  - employee lacks prerequisite skill
  - task requires high psychomotor skills
Self-Teaching Exportable Package
- close supervision not required
- task can be self-learned by individual or groups
- material required for training can be adequate packaged
- do not use if:
  - task failure would result in injury or damage
  - special facilities or equipment required
Formal On-the-Job-Training
- close supervision is required
- task can be self-learned by individual or groups in the workplace
- task decay rate is very high
- do not use if:
  - sufficient equipment is not available for learners to practice on
  - workplace cannot absorb the learners adequately
  - training would be disruptive to normal operations
Classroom
- large group must be taught the same thing
- task difficulty requires a high state of training (task is difficult and requires time to acquire skills)
- learner interaction is required (such as team training)
- material required for training cannot economically be placed in the field
- essential the employee be able to perform upon job entry (high consequence if employees are inadequate performers)
  - do not use if:
  - task may be adequately trained elsewhere

In addition, think blended learning. When I was first trained in ISD we called it BoB (Best of Breed). Blended learning solutions are normally more efficient and effective when designed correctly as they inherit the best of each setting. And do not think of blended as just Brick and Click, but rather any combination of the above, plus more informal options, such as mentoring and social learning media.

ADDIE Backwards Planning Model

ADDIE and the 5 Rules of Zen

ID is not ISD

2011-10-12T07:41:00.000-07:00

One of the trends in the learning industry is proclaiming that a new Instructional Design (ID) model, such as rapid development prototyping, needs to replace Instructional System Design (ISD) because the new model provides more benefits, such as it's newer, dynamic, and faster. Yet ID models differ from ISD models, thus its sort of like saying that a new boat model is going to replace the automobile—yes they are both transportation devices but they do differ in their uses!

ID (Instruction Design) models differ from ISD models in that ISD models have a broad scope and typically divide the instruction design process into five phases (van Merriënboer, 1997):

Analysis
Design (sometimes combined with Development)
Development
Implementation or Delivery
Evaluation

Since ISD models cover a broad spectrum they normally do not go into much detail in the design phase. This is where ID models excel. Since they are less broad in nature and mostly focus on design, they normally go into much more detail for the design phase.

Two popular ISD models are ADDIE and The Dick and Carey Model. ISD can also be extended by using Frog Design's model to solve wicked or complex problems as it aligns with ADDIE:

ISD can be thought more of as a project management tool while ID models are specialized tools used to enhance the learning process.

Omitting ISD and relying strictly on an ID model often omits critical parts of the design process, such as analysis and evaluation. Thus, unless you design for certain groups in an organization or industry in which you know your learners, analysis is important to determine the skill level that the learning program is aimed at. In addition, managers will often identify any performance problem as a training problem, thus the designer needs to ensure it is indeed a training problem rather than a bad process or motivation problem.

Evaluation is not only important to determine if the program is meeting the needs of the organization, but also as a learning tool for the designers themselves.

The best way to use ID models is to plug them into the ISD model as they are needed. For example:

This method allows you to gain the benefit of the ID model that will best suit your needs for enhancing your learning program, while ensuring that your learning program will do what it is supposed to do.

Reference

van Merriënboer, J. J. G. (1997). Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training. Englewood Cliffs, New Jersey: Educational Technology Publications.

A Look Behind Robert Gagnè's Nine Steps of Instruction

2011-08-10T09:45:00.000-07:00

In her post, Questioning Gagnè and Bloom’s Relevance, Christy Tucker describes how we often get caught up in theories without really looking at whether the research supports those theories. In this post, I would like to point out some of the research and newer findings.

While some think the Nine Steps are iron clad rules, it has been noted at least since 1977 (Good, Brophy, p.200) that the nine steps are “general considerations to be taken into account when designing instruction. Although some steps might need to be rearranged (or might be unnecessary) for certain types of lessons, the general set of considerations provide a good checklist of key design steps.”

1. Gain attention

In the military we called this an interest device—a story or some other vehicle for capturing the learners' attention and helping them to see the importance of learning the tasks at hand. For example, when I was training loading and and unloading trailers with a forklift, I would search the OSHA reports for the latest incidence report on a forklift operator who decapitated themself by sticking their head out of the protective structure of the forklift cage in order to get a better view when entering the trailer and then getting it caught between the bars supporting the forklifts protective top and the side of the trailer (it happens more often than we care to think about). This would become the basis for a story on why they needed to pay attention as the forklift may be small, but it weighs several tons and can easily slice off a limb or another body part if not treated with proper respect.

Wick, Pollock, Jefferson, and Flanagan (2006) describe how research supports extending the interest device into the workplace in order to increase performance when the learners apply they new learnings to the job. This is accomplished by having the learners and their managers discuss what they need to learn and be able to perform when they finish the training. This preclass activity ends in a mutual contract between the learners and managers on what is expected to be achieved from the learning activities (this is also closely related to the next step).

2. Tell learners the learning objective

Marzano (1998, p.94) reported an effect size of 0.97 (which indicates that achievement can be raised by 34 percentile points) when goal specification is used. When students have some control over the learning outcomes, there is an effect size of 1.21 (39 percentile points). This is the beauty of using Wick, Pollock, Jefferson, and Flanagan's mutual contract.

Of course the problem that some trainers and instructional designers run into is telling the learners the Learning Objectives word for word, rather than breaking it down into a less formal statement.

3. Stimulate recall

This is building on prior learning and forms the basis of scaffolding by 1) building on what the learners know, 2) adding more details, hints, information, concepts, feedback, etc. 3) and then allowing the learners to perform on their own. Allan Collins John Seely Brown, and Ann Holum (1991) note that scaffolding is the support the master gives apprentices in carrying out a task. This can range from doing almost the entire task for them to giving occasional hints as to what to do next. Fading is the notion of slowly removing the support, giving the apprentice more and more responsibility.

Part of stimulating recall is having the learners take notes and drawing mind maps. Learning is enhanced by encouraging the use of graphic representations when taking notes (mind or concept maps). While normal note taking has an overall effect size of .99, indicating a percentile gain of 34 points, graphic representations produced a percentile gain in achievement of 39 points (Marzano, 1998). One of the most effective of these techniques is semantic mapping (Toms-Bronosky, 1980) with an effect size of 1.48 (n=1), indicating a percentile gain of 43 points. With this technique, the learner represents the key ideas in a lesson as nodes (circles) with spokes depicting key details emanating from the node.

4. Present the stimulus, content

Implement (nuff said)

5. Provide guidance, relevance, and organization

Kind of redundant as it relates to the other steps.

6. Elicit the learning by demonstrating it (modeling and observational learning)

Albert Bandura noted that observation learning may or may not involve imitation. For example if you see someone driving in front of you hit a pothole, and then you swerve to miss it—you learned from observational learning, not imitation (if you learned from imitation then you would also hit the pothole). What you learned was the information you processed cognitively and then acted upon. Observational learning is much more complex than simple imitation. Bandura's theory is often referred to as social learning theory as it emphasizes the role of vicarious experience (observation) of people impacting people (models). Modeling has several affects on learners:

Acquisition - New responses are learned by observing the model.
Inhibition - A response that otherwise may be made is changed when the observer sees a model being punished.
Disinhibition - A reduction in fear by observing a model's behavior go unpunished in a feared activity.
Facilitation - A model elicits from an observer a response that has already been learned.
Creativity - Observing several models performing and then adapting a combination of characteristics or styles.

7. Provide feedback on performance

As Christy's post noted, performance and feedback are good.

8. Assess performance, give feedback and reinforcement

Related to above.

9. Enhance retention and transfer to other contexts

We often think of transfer of learning as just being able to apply the new skills and knowledge to the job, but it actually goes beyond that. Transfer of learning is a phenomenon of learning more quickly and developing a deeper understanding of the task if we bring some knowledge or skills from previous learning. Therefore, to produce positive transfer of learning, we need to practice under a variety of conditions. For more information, see Transfer of Learning.

References

Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator, 6-46.

Good, T. & Brophy, J. (1990). Educational Psychology: A realistic approach. New York: Holt, Rinehart, & Winston.

Marzano, Robert J. (1998). A Theory-Based Meta-Analysis of Research on Instruction. Mid-continent Aurora, Colorado: Regional Educational Laboratory. Retrieved May 2, 2000 from http://www.mcrel.org/products/learning/meta.pdf

Wick, C., Pollock, R., Jefferson, A., Flanagan, R. (2006). Six Disciplines of Breakthrough Learning: How to Turn Training and Development Into Business Results. San Francisco: Pfeiffer

Andragogy vs. Pedagogy

2011-07-07T10:53:00.000-07:00

In his post, Learning is learning, Steve Wheeler asks, “So does the concept of Andragogy add any value to our understanding of learning? For me, the answer is no.”

I would have to disagree because the concept of andragogy has actually added great value to our understanding of learning.

Pedagogy is derived from the Greek words paid meaning “child” and agogus meaning “leader of.” In this pedagogy classroom, the teachers are responsible for all decisions about learning in that they decided what is to be learned, how it is to be learned, when it should be learned, and if it has been learned. Which meant the learners were pretty much in the roles of passive, dependent recipients of the teachers' transmissions. When our public schools were first established, they were based on this pedagogical model.

When adult education was later established, this was the only model at the time, so our profession was also based on it. Which of course lead to a high drop out rate, low motivation, and poor performance. In 1926, Eduard C. Lindereman's book, The Meaning of Adult Education, captures the essence of adult learning:

In this process the teacher finds a new function. He is no longer the oracle who speaks from the platform of authority, but rather the guide, the pointer-out who also participates in learning in proportion to the vitality and relevance of his facts and experiences. In short, my conception of adult education is this: a cooperative venture in nonauthoritarian, informal learning, the chief purpose of which is to discover the meaning of experience; a quest of the mind which digs down to the roots of the preconceptions which formulate our conduct; a technique of learning for adults that makes education coterminous with life and hence elevates living itself to the level of adventurous experiment. - quoted in Nadler, 1984, p.6.4

In the 1950s, European educators started using the term “andragogy,” from the Greek word anere for “adult,” and agogus, “the art and science of helping students to learn.” They wanted to be able to discuss the growing body of knowledge about adult learners in parallel with pedagogy.

Andragogy, is often criticized because as we now know, it also applies to younger learners; however the people behind the theories at the time were trainers of adults rather than educators in the school system, thus they applied their theories to the section of the population that they best knew about. Because of their work, they pioneered the way for the world of pedagogy to also advance itself from being almost entirely passive-based to a more experience-based process of learning.

So yes, Knowles's concept of andragogy is that he intended for it to be different to pedagogy, because pedagogy at the time was extremely passive-based. Just because pedagogy is finally catching up to andragogy is not a strong enough reason to drop the concept from our terminology. I believe we should be embracing the term because of its rich history and pioneering the way of our present concept of learning.

Reference

Nadler, Leonard (1984). The Handbook of Human Resource Development. New York: John Wiley & Sons

Marching Backwards into the Future

2011-06-27T10:34:00.000-07:00

A recent post by Bersin & Associates notes, “Approximately three-quarters of employers globally cite a lack of experience, skills or knowledge as the primary reason for the difficulty filling positions. However, only one in five employers is concentrating on training and development to fill the gap. A mere 6% of employers are working more closely with educational institutions to create curriculums that close knowledge gaps.”

Doh! These same employers slashed their work forces for a total job loss of 8,700,000 jobs since the recession started in December 2007. Only 4,444,000 jobs have been added since then, which leaves a net loss of ~~425,6000~~ 4,256,000 jobs.

What were they thinking? That they could slash their “most valuable asset” and when the economy picks back up, find the knowledge and skills they require? Yep—short term thinking at its best—and of course it backfired in this complicated/complex work environment.

In a prior post I wrote about the three most important words that managers in an organization must know when it comes to learning—training development, and education.

Training is learning that is provided in order to improve performance on the present job, which means it's orientated towards the present. What these employers should have been thinking is towards the future—what skills and knowledge are we going to need when the economy picks back up? Which means they should have implemented development and education processes.

Development is training people to acquire new horizons, technologies, or viewpoints. It enables leaders to guide their organizations onto new expectations by being proactive rather than reactive. It enables workers to create better products, faster services, and more competitive organizations. It is learning for growth of the individual, but not related to a specific present or future job.

The past went that-a-way. When faced with a totally new situation, we tend to always to attach ourselves to the objects, to the flavor of the most recent past. We look at the present through a rear-view mirror. We march backwards into the future. - Marshall McLuhan

As we craft our learning processes we must remember the three most important words and ensure that our clients/customers also understand them. Failure to do so will again result in marching backwards into the future.

Five Years later: A Review of Kirschner, Sweller and Clark's Why Minimal Guidance during Instruction Does Not Work

2011-06-07T14:49:00.000-07:00

After having a short discussion with Guy Wallace on his blog, I decided to do a review of Kirschner, Sweller and Clark's, Why Minimal Guidance during Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching, in which they postulate that students who learn in classrooms with pure-discovery methods and minimal feedback often become lost and frustrated, and their confusion can lead to misconceptions.

The paper caused a bit of a stir in the learning and training community when it was published five years ago, especially among those who lean towards a more constructivist approach. However, while the author's critics did raise some good points, the paper is a good reminder that learning and training professionals often carry new ideals and technologies to the extreme. For example:

We had the visual movement from about 1900—1950, which brought us Dale's Cone of Experience. And of course someone had to add some bogus percentages to it to make it more “official.”
When VCR's arrived we made training tapes of everything… even if it did not make sense.
eLearning was supposed to kill the classroom.
Formal and informal learning were supposed to be at odds which each other, even though each hour of formal learning spills over to four-hours of informal learning.
All learning is social! Uhh… no. While the majority of learning may be social we often still learn things on our own.

Thus Kirschner, Sweller and Clark's paper is an important reminder for us to not carry Problem Based Learning (PBL) to its extreme. That is, while it has its strengths, learners often need a more direct approach in order to build a solid foundations before being presented with PBL.

With that being said, we do need to take a closer look at the paper. For those that are interested, there is a list of papers that discuss the Direct Instruction versus Constructivism Controversy (they are located at the bottom of the page).

The Title and Paper gives Little Respect to the Constructivism Approach

With the title blaring, “Why Minimal Guidance during Instruction Does Not Work” rather than, “Why Minimal Guidance during Instruction Does Not Work for Novice Learners,” the authors almost seem to ignore that PBL is a necessity in order to promote deeper levels of understanding. They do pay some respect to constructivism, such as:

Higher aptitude students who chose highly structured approaches tended to like them but achieve at a lower level than with less structured versions

Certain aspects of the PBL model should be tailored to the developmental level of the learners… there may be a place for direct instruction on a just-in-time basis. In other words, as students are grappling with a problem and confronted with the need for particular kinds of knowledge, a lecture at the right time may be beneficial.

However, they end up admonishing constructivist:

According to Kyle (1980), scientific inquiry is a systematic and investigative performance ability incorporating unrestrained thinking capabilities after a person has acquired a broad, critical knowledge of the particular subject matter through formal teaching processes. It may not be equated with investigative methods of science teaching, self-instructional teaching techniques and/or open-ended teaching techniques. Educators who confuse the two are guilty of the improper use of inquiry as a paradigm on which to base an instructional strategy.

But it seems, at least to me, they may be doing the same, but only at the opposite end of the continuum. For example, they seem to treat their theories as laws, yet…

Cognitive Load Theory Coming Under Withering Attacks

The paper relies heavily on Cognitive Load Theory, yet we have to realize that it is still a theory rather than a law. Will Thalheimer lists several papers on his site that raises several concerns about Cognitive Load Theory. For example, even though we know that working memory can only hold about seven chucks (which actually may only be four, give or take one), using the old KiSS (Keep it Simple Stupid) principle can be just as effect because trying to count the number of chunks can be quite difficult, if almost impossible. For example, how many chunks are in Rene Descartes statement, “I think, therefore I am?”

Thus, both the authors and the constructivism movement are guilty of jumping on theories before they are fully understood. But why do we do this? Joel Michael writes in Advances in Physiology Education:

…it is important to recognize that educational research is difficult to do; this has been cogently highlighted by Berliner (8) in "Educational research: the hardest science of them all." Berliner points out that unlike a physics experiment, in which it is possible to readily distinguish between the independent and dependent variables, and also possible to isolate and control all of the independent variables, in educational experiments all of this is problematic. Researchers may not agree on which variable is the dependent variable of greatest interest or importance. There may be disagreements about which independent variable(s) are to be manipulated. There may be disagreements about how to measure any of the relevant variables. And, finally, it may be extremely difficult, or even impossible, to isolate and manipulate all the variables suspected of being involved in the phenomena being studied.”

Rather than waiting for eons to pass before all the research is available, we (the learning, training, and educational community) often jump into a new theory because will simply do not want to wait until we are dead and buried before we can fix and/or improve our methodology. With that in mind…

Evidence for Constructivism

Joel Michael continues his discussion for promoting active learning with these two studies:

1. Support for discovery learning is provided by a study in which students engaged in a course that incorporated some discovery learning exercises were tested, and their performance on questions related to topics learned through discovery learning was compared with their performance on questions related to topics learned in lecture (Wilke, Straits, 2001). The authors concluded that performance was better on those topics learned through discovery learning.

2. Burrowes compared learning outcomes in two sections of the same course taught by the same teacher. One section was taught in the traditional teacher-centered manner (control group of 100 students), whereas the other section was taught in a manner that was based on constructivist ideas (experimental group of 104 students). The results of this experiment were striking: the mean exam scores of the experimental group were significantly higher than those of the control group, and students in the experimental group did better on questions that specifically tested their ability to “think like a scientist.” Reference: Burrowes PA. Lord's constructivist model put to a test. Am Biol Teacher 65: 491–502, 2003.

While you can find plenty of other research findings on constructivist methods, the ideal that you can teach learners to “think like scientists” is fascinating because problem solving skills are extremely hard to train. That is, conduct a problem solving course in an organizational setting and you will more than likely get little or no results. It's almost as if the process must be embodied within the discipline.

Embodied Cognition

On the Brain Science Podcast, Ginger Campbell discusses Embodied Cognition with Lawrence Shapiro (both podcast and transcript can be found in the link). They note that in cognitive science, the brain is normally studied while isolated from the world and from the body. While in contrast, embodied cognition imagines not that the brain can be isolated from the body and the environment, but thinks of the body as in some sense shaping, or constraining, or involved in the very processing of the kinds of information that an organism needs to interact successfully with the world.

In the podcast, Dr. Shapiro talks about a fascinating work on the use of gesture. He notes that boys perform better in certain spatial reasoning tasks than girls. When psychologists studied this they've noticed something kind of interesting—boys rely on gestures a lot more than girls do when solving spatial reasoning tasks. Boys use gestures to work out the problem, at the same time they're talking through the problem. And often, they don't synchronize with the verbalizing. It's as if they have two different systems working at the same time—one a gesture system, and one a verbalization system. Gestures seem to be a part of the process of figuring out these spatial reasoning tasks.

They also discuss the study of kittens moving around their environments by pulling carriages with other kittens in them. And the ones in the carriages presumably see everything that the kittens pulling them see, but because they don't actually employ their motor systems to move them around the environment, their perceptual systems don't develop properly. The idea there seems to be that part of what's necessary for perception is actual exploration of the environment; not just being a passive recipient, like these kittens conveyed in the carriages are.

Thus rather than focusing on pieces of conceptually unrelated pieces of information, such as practice first and then learn problem solving second, perhaps we should be focusing our learning processes on entire ideas and concepts whenever possible.

I would interested in your thoughts on the subject by leaving a comment, creating a blog post, or through Twitter (I'm @iOPT).

Training at its Basic is a Positive Impact Caused by Learning

2011-06-02T08:05:00.000-07:00

The WSJ recently ran an informative article, Lessons Learned that discusses the importance of the need to create a workplace environment that actively encourages people to change. They start their article with the statement, “With some studies suggesting that just 10% to 40% of training is ever used on the job, it is clear that a big chunk of the tens of billions of dollars organizations spend annually on staff development is going down the drain.”This is actually a myth as some studies suggest the transfer rate is actually around 60% as the the study the WSJ used is based on extremely faulty research (see Myth - 10% of Training Transfers to the Job). However, 60% is still too low of a transfer rate, thus we know we must use a better method for designing our learning processes.

To ROI or not to ROI?

The article notes that an effective post-training follow-up activity is the performance assessment—“When employees know that they are going to be observed and given feedback on their performance, the motivation to use newly learned skills and knowledge increases.” This means you must know what you are going to assess before you design the training, which in turn means the learning process must be based on a goal achieved through backwards planning that will have a positive impact upon the organization.

Does this mean you need to perform a ROI? No. Learning and Training departments normally only need to provide an ROI if they are the initiators of a learning or training process. For example, if you presently outsource your Microsoft Office training program and you want to bring it in-house because you believe you can do it cheaper and better, then you should provide an ROI to show this to be true. If on the other hand you will provide the training for the users of a new computer system, then it is up to the original initiators, normally the IT department, to provide the ROI.

In other cases... it depends. For example if a manager comes to you for a request for training that will eliminate a problem in her department and you determine that training is indeed the answer, then you will have to decide if an ROI is needed or not. In many cases the manager simply wants the problem to go away. And yes, training may be more expensive than solving the problem, but this frees the manager to help grow the organization rather than spend her time putting out fires. You are really investing in her—by eliminating the problem you allow her to spend more time growing the organization, thus the training will pay off in the long run. One rule of thumb is to provide an ROI whenever possible to show that your efforts have a positive financial impact on the organization, but always keep in mind that even if the cost savings are not there in the short turn, it could pay off in the future.

Getting the Impact out of Training with Agile Design

So why you may or may not have an ROI, you still need a positive result or impact. Nadler defines “training” as learning that is provided in order to improve performance on the present job (1984). Most definitions closely follow Nadler on these two points: 1) training always involves learning and 2) performance is improved. Thus training is basically a positive impact caused by learning. If it does not follow these two points then that means you are doing something besides training. It doesn't mean its wrong or right, but it is simply not training.

While the author of the WSJ brings the learners more into the learning and training process, there is still another step to go—include them in the design. Rittel (1972) noted that the best experts with the best knowledge for solving wicked problems are often those affected by the solution; in this case it is the learners themselves. Yet, the only time we normally bring them in is to be guinea pigs for testing our learning process. While you might not be able do bring the entire population of them in on the planning stage, we do need bring in enough learners who will actually represent the population.

This is the heart of Agile Design. The learners are the real stakeholders, even if you or the managers don't agree to what they are saying, you need to listen, guide, and act on their needs and perspectives so that they take ownership of the learning and performance solution. In addition, they gain metalearning and metacognitive skills.

This is true “Learner Design.” Simply designing a learning process for them is andragogy or pedagogy design. True learner designed process involves the learners. Change works best if the people it affects are involved in the process. Learning is no different if you want to change performance on the job. Involve the learners so you not only make them part of the change process, but also as Rittel noted—they are often the experts who can provide good advice.

References

Nadler, Leonard (1984). The Handbook of Human Resource Development. New York: John Wiley & Sons.

Rittel, H. (1972). On the planning crisis: systems analysis of the “first and second generation.” Bedriftsokonomen. No. 8, pp.390-396.

Creating and Evaluating Informal & Social Learning Processes in a Call Center

2011-05-23T06:52:00.001-07:00

I recently received this comment on my post, Using Kirkpatrick's Four Levels to Create and Evaluate Informal & Social Learning Processes:

“What do you do when the "learners" are new hires? And the "environment" is a real-time call center?”

Using the same process as in the last post (shown below), we start off with the main goal or objective:

Kirkpatrick's Backwards Planning and Evaluation Model

1. Results or Impact - What is our goal?
2. Performance - What must the performers do to achieve the goal?
3. Learning - What must they learn to be able to perform?
4. Reaction - What needs to be done to engage the learners/performers?

1. What is our Goal?

Training new hires is normally performed because proficient ones cannot be recruited. However, using training as the only performance solution is not a good choice as it is normally one of the more costly and time-consuming solutions if done correctly. Thus, when formulating your goal, don't think of training as being the solution or goal, but rather what are the benefits you are looking for. For example:

Our goal is to convert interested callers into extremely satisfied and delighted customers. We will achieve this by providing timely, accurate and professional service at each and every customer contact and answering their questions and inquiries in a timely and professional manner.

The benefit of our goal is to maintain/increase customer satisfaction, which will lead to higher sales.

2. What must the performers do to achieve the goal?

While there are several tasks the employees should be able to perform, a few of them that would lead to higher sales are:

Greet customers in a timely, cheerful and professional manner
- The benefit is to jump start the customers' experience from the moment they call
Quickly and accurately find product information
- The benefit is to show our customers that we are professionals who will take care of their needs
Understand the culture, mission and policies of the company in order to make wise and timely decisions
- The benefit is to not only provide customers with our goods and services, but to also show them we can aid them with difficult problems

3. What must they learn to be able to perform?

In this example we need a Learning Environment (not just training) that will enable new hires to perform correctly in our call center so that it can perform its mission:

Star Diagram of the Continua of Learning

click image for a larger version

Note: for more information on the above diagram see:

A number of experiences and activities are then designed for the learning environment that will enable the Customer Service Representatives to perform the three tasks:

Task One: Greet customers in a cheerful and professional manner

Social Learning: The learners will discuss with each other what makes a great Customer Service Representative.

Role Play: This activity will be performed in the classroom where the learners will take turns playing customers and Customer Service Representatives. When a learner is role playing the customer, he or she will be provided a number of scenarios that range from a happy to dissatisfied customer (some sample role playing activities may be found here).

Task Two: Quickly find product information

elearning: Explains the company's database and how to find product and service information.

eLearning Branching Scenarios: This course will take the learners through a number of scenarios for finding information that a customer requests.

Informal Learning: The learners are coupled with experienced employees in order to gain real experience.

Social Learning with Social Media: Employees are connected to a micro-blogging service (e.g. Yammer or Twitter) so that they may ask for and pass on information through a social network.

Task Three: Understand the culture, mission, and policies of the company in order to make wise and timely decisions

eLearning Branching Scenarios: This course is an extension of the last eLearning Branching Scenario in that once a learner finds information that customer requests, he or she then has to go through various scenarios to help the customer make a decision.

Informal Learning: The learners are coupled with experienced employees in order to gain real experience.

Social Learning with Social Media: Employees are connected to a micro-blogging service so that they may ask for and pass on decision making techniques through a social network.

Social Learning with Social Media: Employees are provided a blogging platform that will enable them to find and pass on decision making techniques that may require more detailed information than the micro-blogging service allows — the micro-blogging service is for quick and short bursts of information while the blog is for more complex and detailed information.

Wiki: For storing and retrieving lessons learned.

Note that the learning platform may start with traditional classroom training, but it is blended with elearning and informal learning. In addition it is transformed into a true learning process, rather than an event, in that it is implemented into their daily work flow so that they can continue to not only learn, but help others learn.

4. What needs to be done to engage the learners/performers?

Before the learners enter the learning environment, each learner's respective manager will ensure that the learner understands the importance of the training they are about to receive. In addition, the learners and managers will set goals and discuss potential problems. After the initial elearning and classroom learning programs are completed, the manager will follow-up with the learners and assign them coaches/mentors and follow their progress on a weekly basis.

Evaluating the Learning Platform

Since we know precisely what each part of the Learning Platform was designed to perform, our task of evaluating the program becomes much easier:

Kirkpatrick's Backwards Planning and Evaluation Model

1. Results or Impact (What is our goal?) Did we achieve higher customer satisfaction? This can also be tied to a hard ROI , such as an increase in sales.
2. Performance (What must the performers do to achieve the goal?) Can the employees now perform as expected?
3. Learning (What must they learn to be able to perform?) This is assessed in the elearning programs and discussions with the experienced employees involved in the informal learning sessions.
4. Reaction (What needs to be done to engage the learners/performers?) The learners managers can provided input on the level of the learner's reaction and engagement of the learning platform.

1. Results or Impact (What is our goal?)

Did we achieve higher customer satisfaction? This can also be tied to a hard ROI , such as an increase in sales.

2. Performance (What must the performers do to achieve the goal?)

Can the employees now perform as expected?

3. Learning (What must they learn to be able to perform?)

This is assessed in the elearning programs and discussions with the experienced employees involved in the informal learning sessions.

4. Reaction (What needs to be done to engage the learners/performers?)

The learners managers can provided input on the level of the learner's reaction and engagement of the learning platform.

This backwards planning process can help you pinpoint problems. For example, let's say that you do not get an increase in sales. That means to go back one step and see if the learners are performing as desired. If they are, then that means your initial premise was wrong (greater customer satisfaction does not lead to higher sales) or something else is preventing it, such as your product is priced too high.

On the other hand, if they are not performing as desired, then you have to evaluate the working environment to determine if something is preventing the learners from using their skills, such as processes that are counter-productive to great customer service. If you determine that they should be able to perform, then evaluate the learners to see if they can perform or if something in the learning process is preventing them from learning, such as not enough practice time.

If the learning process is sound, then go back one more step and determine if the learners are engaged. That is, do the have the basic skills that will allow them to master the learning program and/or do they have the motivation and desire to complete the learning program (maybe they see it as a waste of time).

Using Kirkpatrick's Four Levels to Create and Evaluate Informal & Social Learning Processes

2011-02-22T07:29:00.000-08:00

In my last post, The Tools of Our Craft, I wrote how the Four Level Evaluation model is best used by flipping it upside down and that it can be used to evaluate informal and social learning. In this post, I want to expand on the second point—evaluating informal and social learning.

Backwards Planning and Evaluation Model

1. Results or Impact - What is our goal?
2. Performance - What must the performers do to achieve the goal?
3. Learning - What must they learn to be able to perform?
4. Reaction - What needs to be done to engage the learners/performers?

Inherent in the idea of evaluation is “value.” That is, when we evaluate something we are trying to make a judgment about the worth of it. The measurements we obtain gives us information to help base our judgment on. This is the real value of Kirkpatrick's Four Level Evaluation model as it allows us to take a number of measurements throughout the life span of learning process in order to place a value on it, thus it is a process-based solution rather than an event-based solution.

Each stakeholder will normally only use a couple of the levels when making their evaluation, except for the Learning Department. For example, top executives are normally only interested in the first one—Results, as it directly affects the bottom-line. Some are also interested in the last one, Reaction, as they are interested in the engagement aspect—are the employees engaged in their job? Managers and supervisors are normally most interested in the top two levels, Results and Performance, and somewhat in the last one, Reaction. While the Learning Department needs all four to properly deliver and evaluate the learning process.

Note that this post uses as actual problem that is based on informal and social learning for the solution. I wrote about part of it in Strategies for Creating Informal Learning Environments, thus you might want to read the first half of it (you can stop when it comes to the section on OODA).

Results

Implementing a learning process is based on what results or goals you are trying to achieve—and identifying what measurements you need to help evaluate your results will help you to zero in on identify the result or goal you are trying to achieve. For example, saying that you want your employees to quickly find information is normally a poor goal to shoot for as it is hard to measure. Starting with a focused project and then letting demand drive additional initiatives is normally the best way to start implementing social and informal learning processes.

If you find that you are unable to come up with a good measurement, that normally means you have not zeroed in on a viable goal. In that case, use the Japanese method of asking “Why?” five times or until you are able to pinpoint the exact goal you are trying to achieve. Establishing new or improving learning/training processes normally begin with a problem, for example, a manager complains that when he reads the monthly project reports he finds that employees are often faced with the same problems as others and in turn, repeat the same learning process again, thus the same mistakes are repeated throughout the organization.”

“Why?”

“No one realizes that others within the organization have had the same problem before and have normally documented their solution (Lesson Learned).”

“Why?”

“There is no central database for them to look and the people who work next to them are normally unable to help.”

Zeroing in on the actual cause of the problem helps you to build a focused program, in this example, it's a central database for “Lessons Learned” and a means of connecting the people within the organization to see if anyone has been faced with the same problem (and vice versa—allowing people to tweet (broadcast) their problems and their solution that may be of help to others).

In addition, you now have a viable measurement—counting the number of problems/mistakes in the project reports each month to see if they improve.

Performance

In a normal training situation, performance on the job is normally easily evaluated. For example, when I instructed forklifts operations in a manufacturing plant, after the training/practice period we would assess the learners performing on the forklifts in the actual work environment to ensure they could operate safely and correctly under actual working conditions.

In addition, I used to train users to use Query/400 (a programming language to extract information from a company's computer system). One of the methods we used to assess the performers was that when they returned to their workplace, we required them to build three queries that were assessed by someone from the training department to ensure they could perform on the job. Thus the training is transformed from an event to a process by ensuring their skills are carried over to the workplace.

However, in our working example, it would be hard to observe the entire “Lessons Learned” process as it is a three-prong solution that uses informal and social learning:

Capture the Lessons Learned by using an After Action Review (AAR)
Store it in a social site (such as a wiki or SharePoint) for easy retrieval
Provide a microblogging tool, such as Yammer, to help others to ask about lessons learned that might pertain to their problems and to tweet lessons learned

The first part of the solution could somewhat be evaluated by observing some of the AARs and watching the informal learning taking place as they discuss their problems and solutions, however, the second and third points would be difficult as it would be hard to sit at someone's desk all day to see if they are using the wiki and microblogging tools. While there are probably a number of solutions, one method is to identify approximately how often the social media tools should be used on a daily, weekly or monthly basis and then determine if expectations are being met by counting the number of:

contributions per month to the wiki (based on their Lessons Learned in the AAR sessions)
contributions to the microblogging tool (short briefs on their Lessons Learned)
questions asked on the microblogging tool by employees who could not find a Lesson Learned in the wiki that matched their problem

The approximations are based on the number of problems/mistakes found in the project reports and the total number submitted. You might have to adjust your expectations as the process continues, but it does give you a method for measuring the performance. Note that Tim Wiering has a recent blog post on this method in the Green Chameleon blog.

In addition, once the performers have started using the tools, you can interview them by asking how the new tools are helping them and then capture some of the real success stories, such as videotaping them or using a question and answer interview and then blogging about it. These stories have a two-fold purpose:

The stories themselves are evidence of the success of the performance solution.
The stories can be then be used to help other learners/performers to use the new tools in a more effective manner as stories carry a lot of power in a learning process because the learners are able to identify and relate to them.

Learning

First, the purpose of this level is not to evaluate what the performers are learning through the AARs, microblogging, and wiki tools when they return to their job (that measurement is captured in the Performance Evaluation), but rather what they need to learn so that they can use the tools on the job. Look around almost any organization and you see processes, programs, tools, etc. that were built on the idea that if we build it, they will come, but are now wastelands because the performers saw no use for them and/or had no real idea how to use them. Just because a tool, such as Yammer or Twitter may be obvious to you, does not mean the intended performers will see a use for it, and for that matter, know how to use it.

In addition, while one organization may not care if someone sends an occasional tweet about the latest Lady Gaga video, another may frown on it, so ensure the intended performers also know what not to use the new tools for.

Since these learning programs can be elearning, classroom, social, informal, etc. and the majority of Learning Designers know how to build and evaluate them, I'm not going to delve into that in this post.

Reaction Engagement

While this may be the last level when flipping Kirkpatrick's Evaluation Model, it is actually the foundation of the other three levels.

One of the mistakes Kirkpatrick made is putting too much emphasis on smiley sheets. As noted in the excellent article, Are You Too Nice to Train?, measuring reaction is mostly a waste of time. What we really want to know is how engaged the learners will be in the learning level and will that engagement carry through to the performance level. People don't care so much about how happy they are with a learning process, but rather how will the new skills and knowledge be of use to them?

For example, when I was stationed in Germany while in the Army we trained on how to protect ourselves and perform during CBR (Chemical/Biological/Radiological) attacks. One of the learning processes was to don our CBR gear (heavy clothing lined with charcoal to absorb the chemical and biological agents, rubber gloves, rubber boots, the full-face rubber protective mask, and of course our helmets to protect our heads) in the midday heat of the summer time and then using a compass and map, move as fast as we could on foot to a given location about two miles away. And I can tell you from experience, this is absolutely no fun at all, yet we learned to do it because no one wants to die from a chemical or biological agent—a ghastly way to go. Thus the training had us totally engaged even though the training was absolutely horrible.

Thus the purpose of this phase is to ensure the learner's are on board with the learning and performance process, which is often best accomplished by ensuring a portion of them and their managers are included in the planning process. You need the managers to help ensure they are on board as employees most often do what their managers emphasize (unless you have some strong informal leaders among them).

Reversing the Process

By using the four levels to build the learning/performance process (going through levels 1 to 4 in the chart below), it is now relative easy to evaluate the program by reversing the process (going through the levels in reverse order [4, 3, 2, 1]:

Evaluation Level	Create	Measurement/
1. Results or Impact - What is our goal?	Implement a process that allows the employees to capture Lessons Learned so that others may also learn from them when similar problems arise.	Reduce number of repeated problems/mistakes in the project reports by 90%.
2. Performance - What must the performers do to achieve the goal?	Identify and capture “Lessons Learned” in an AAR, post them on a wiki, and tweet them using Yammer. When problems in their projects arise, they should be able to search the wiki and/or use Yammer to see if there is a previous solution.	Count the : contributions per month to the wiki contributions to the microblogging tool questions asked on Yammer Interview performers to capture success stories.
3. Learning - What must they learn to be able to perform?	Perform an AAR. Upload the captured “Lessons Learned” to a wiki. Search and find documents in a wiki that are similar to their problem. Microblog in Yammer.	Proficient use of an AAR is measured by using Branching Scenarios in an elearning program and performing an actual AAR in a classroom environment. The proficient use of the wiki and Yammer are measured in their respective elearning program (multiple choice) and by interacting (social learning) with the instructor and other learners on Yammer.
4. Reaction - What needs to be done to engage the learners/performers?	Bring learners in on the planning/building process to ensure it meets their needs. Managers will meet with the learners on a one-on-one basis before they begin the learning process to ensure the program is relevant to their needs. The instructional staff will meet with the learners during the learning process to ensure it is meeting their needs. The managers, with help from the learning department, will meet with the performers to ensure the new process is not conflicting with their daily working environment.	Learner/performer engagement problems/roadblocks that are encountered will be the first item discussed and solved during the weekly project meetings.

Since we started with a focused project, we can now let demand drive additional initiatives that expand upon the present social and informal learning platform.

How do you build and measure learning processes?

The Tools of Our Craft

2011-02-13T12:19:00.000-08:00

The latest edition of Chief Learning Officer magazine contains an interesting article, Time's Up (you can also read the article on the author's blog). It is about Donald Kirkpatrick's Four Level Evaluation Model that was first published in a series of articles in 1959 in the Journal of American Society of Training Directors (now known as T+D magazine).

The author, Dan Pontefract, sums up the article in his last statement, “Diverging from the cockroach, it's time for the learning profession to evolve.” While Dan's article is thought-provoking, I believe it misses out on two points, 1) since it is old, it's no good and 2) it has not evolved.

Old Does Not Mean Outdated

Interaction Design is closely related to our craft, training and learning. While the concept of interaction design has been around for ages, it was not formerly defined until 1990 by Bill Moggridge, co-founder of the Silicon Valley-based design firm IDEO. While it is one of the newer design professions, it still relies on older tools. For example, one of the tools used is the affinity diagram that was developed by Jiro Kawakita in the early 1960s. Thus, being old does not mean a tool should be extinct. If that was true, the cockroach would have disappeared millions of years ago, yet, because it has evolved, it has managed to survive... much to the disgust of anyone who has had their home invaded by them.

enso circle by Vibhav

“Nature itself is full of beauty and harmonious relationships that are asymmetrical yet balanced. This is a dynamic beauty that attracts and engages.” - Garr Reynolds

While people who have had their homes invaded by the cockroach look at them in disgust, entomologists look at them as one of the marvels of natures. Learning/Instructional Designers should not look upon our tools, such as ADDIE and Kirkpatrick's model as disgusting objects that have invaded our craft, but rather more as entomologists look upon the lowly cockroach—marvels of our craft that have survived the test of time.

The Evolution of Our Tools

Just as the ADDIE model has evolved over time, Kirkpatrick's model has also evolved. One of its main evolutionary steps was flipping it into a backwards planning model:

Results or Impact - What is our goal?
Performance - What must the performers do to achieve the goal?
Learning - What must they learn to perform?
Reaction - What needs to be done to engage the learners/performers?

While I blogged of this in 2008, Flipping Kirkpatrick, Kirkpatrick himself wrote of this several years earlier. This method align's with how Dan's article says we should start, “start with an end goal to achieve overall return on performance and engagement.” In addition, this in no way treats learning as an event, but rather a process. What is interesting, is how closely Kirkpatrick's evolved model fits in with other models, such as Cathy Moore's Action Mapping.

Like the ADDIE model, Kirkpatrick's model is often called a process model. However, this is only true if you blindly follow it. If you remove your blinders and study and play with it, it becomes a way to not only implement formal learning, but informal, social, and nonformal learning as well. For example, in step three, What must they learn to perform?, does not imply strictly formal learning methods, but rather any combination of the four learning processes (social, informal, nonformal, and formal).

How do you see our tools evolving?

Star Diagram of the Continua of Learning

2011-01-02T13:49:00.000-08:00

In my last post, Should the Door be Closed or Open, Nick Kearney commented that the star diagram was a better representation than the various continua I laid out. I agree; however, since the star diagram is composed of continua, I think when discussing a particular one, as I did in the last post, it helps to just show the one being discussed.

As show in the diagram below, I did some adjustments to it (you can click the diagram for a larger version):

Star Diagram of the Continua of Learning

As I noted in my last post, I put social learning and reflection on the same continuum (The Door), as the real purpose is that sometimes we need to be alone with our thoughts, while at other times we need to interact with others. And of course there are a lot of alternatives between the two (social reflection being one of them).

I also dropped the Purpose of Learning Continuum (intentional and incidental/serendipitous). While it is an interesting concept, I don't believe that it fits in with the diagram in that it does not help us to design better learning/performance platforms.

David Winter (@davidawinter) tweeted me with the suggestion of adding another continuum: Impact - 'reinforcement/augmentation' of existing understanding/behaviours/identity vs 'transformation.' After thinking about it, I believe it belongs on the Workflow Continuum (EPSS/performance support and training). However, I'm not sure which of the terms are better. I'm thinking that it should be called the 'Workflow Continuum' with augmentation on one end and transformation on the other. I believe that EPSS/performance support and training would be some of the options that lie between the two:

What are your thoughts?

Should the Door be Closed or Open: Thoughts on the Social Learning and Reflection Continuum

2010-12-30T21:10:00.000-08:00

I have been doing some more reflecting on the Social Learning and Reflection Continuum. This reflection includes both social reflection (mostly via Twitter) and self-reflection. Part of it includes how the continuum fits in with other continuums and does the Social Learning and Reflection Continuum really make sense? I created the diagram below to see what would happen if I changed it and how other forms of learning fit in. If you click the diagram it will bring up a larger diagram in a new window:

In the diagram I replaced the Social Learning and Reflection Continuum with Social Learning and Achorite. This was mostly because of some tweets with Marcia Conner (@marciamarcia) where she thought that Solitude should be on the opposite side of Social Learning. I did not really like that word so I used the term anchorite, which in part means “rural countryside” in order to contrast it to social, as meaning village or town” (e.g. it takes a village or number or people).

NOTE: While the opposite ends of the various continuums discussed below may be a choice of one or the other, they are more often shades of various degrees and/or combinations. That is, don't just look at the ends, but also picture a wide rage of possibilities inside of them.

The Door

I had the thought, “What is the real purpose of the continuum? Is it to count the number of people involved? Or does it have a deeper meaning.” Then I remembered of what I read when the physicist Freeman Dyson commented on the subject. He noted that when writing, he closes the door, but when doing science, he leaves it open. This is because when writing you need to perform deep reflective thoughts, but when doing science you welcome being interrupted because it is only by interacting with other people that you get anything interesting done.

Thus it is not really about the number of people involved in the learning episode, but rather, do you welcome the thoughts of others or do you need to sort out your thoughts and ideas without being interrupted? Counting the number of people in a learning episode does not make sense as it sort of like counting the number of seats in formal learning — who cares? The real purpose is do you need to discuss ideas with others or do you need to sort ideas, order them, toss out invalid ones, etc. within your own mind.

So I named this continuum “the door” – do you need it open or shut during a specific point when learning? And of course you may decide to choose a combination and have Social Reflection — engaging with others in a way that encourages talking with, questioning, or confronting, to aid the reflective process by placing the learner in a safe environment in which self-revelation can take place.

Thus, I am now sticking with my initial premise that reflection belongs on the same continuum with social learning.

Direction of Control

David Winter (@davidawinter) thought perhaps that Autonomous should be placed opposite of Social Learning. At first I thought his term was better than Achorite or Solitude. But then I thought some more and decided that Guided Learning was really its opposite, thus in the first diagram above I have them on their own continuum. But when I started to name them, it dawned on me that they had the same purpose as the Formal and Informal Continuum — who controls the learning? Since Autonomous and Guided Learning has slightly more precise meanings than Formal and Informal, I placed them on the inside of the Direction of Control Continuum.

Known or Unknown?

Collaborative Learning is quite similar to cooperative learning in that the learners work together in teams to increase their chance of deeper learning. However, it is a more radical departure from cooperative learning in that there is not necessarily a known answer. For example, trying to determine the answer to "how effective is reflection?" would be collaborative learning as there are a wide ranges of possibilities to this question, depending upon the learners' experiences and perspectives.

Purpose of Learning

I included a Purpose of Learning Continuum as learning normally has a purpose during an informal or formal episode, but often we learn something that was not in the initial plan. However, that learning may prove later to have a real and important purpose.

Processing

The processing continuum is important because it determines how we will learn something. If it is easy to learn, we may only have to listen, observe, feel, etc. But as it becomes more complicated, we need to actually do it. Of course that is not always possible, so between the two ends of the continuum are the various activities that we may practice in order to be able to perform in a real work setting.

Workflow

While writing this post, I thought of another learning continuum, workflow — can the learning be embedded within the learner's workflow or does it call for a training process?

Thoughts?

What are your thoughts on these various learning continuums? Are there more? Do these make sense? Please let me know by leaving a comment, Twitter me (I'm @iOPT), or carrying the discussion further on your own blog (send me a tweet so I can RT it).

The Social Learning and Reflection Continuum

2010-12-29T09:25:00.000-08:00

Recently I had a Twitter conversation with Marcia Conner (@marciamarcia), Aaron Silvers (@mrch0mp3rs), and David Winter (@davidawinter) on the subject of reflection. Or more specifically, is reflection on the same continuum as social learning:

Being on opposite sides of the continuum does not mean it's one or the other, but rather there are different degrees and combinations of social learning and reflection. The reason I place them on the same learning continuum is that their definitions seem to be just about the opposite (one is performed with others while the other is often performed in one's head):

Social Learning: a process of learning caused or favored by people being situated in a common environment and interacting and observing one another. This allows the learners to not only perceive each other for comparison and self-evaluation, but also see others as a neutral source of information, which may help or speed several forms of instrumental learning.
Reflection is thinking for an extended period by linking recent experiences to earlier ones in order to promote a more complex and interrelated mental schema. It normally involves looking for:
- commonalties
- differences
- interrelations beyond their superficial elements

The middle of the learning continuum might be termed Social Reflection: engaging with another person in a way that encourages talking with, questioning, or confronting, in order to aid the reflective process by placing the learner in a safe environment in which self-revelation can take place (Hatton, Smith, 1995):

And of course we can combine the Learning Continuum with other continuums to form a quad:

Note: examples of tools that promote reflection

How do you view the Social Learning and Reflection Continuum?

Designing eLearning

2010-12-03T09:51:00.001-08:00

A learning methodology is a set of procedures composed of methods, principles, and rules for enhancing individual capacity and performance. Yet, some elearning designers only think of the technology and content, which normally leads to a “page-turning” design — the learner reads what is on the screen and then clicks the next button. While this can bring about knowledge, which is important, the design often fails to follow-up with the next step — performance — having the learners practice the skills in order to master them. While there are a number of means of achieving this, one option is using a design architecture composed of the “Five Types of Content in eLearning” (Clark, Mayer, 2007) and the six categories of Bloom's Revised Taxonomy:

Five Types of Content in eLearning

Fact - unique data (e. g., symbols for Excel formula)
Concept - a category that includes multiple examples (e. g., Excel formulas)
Process - a flow of events or activities (e. g., how a spreadsheet works)
Procedure - step-by-step task (e. g., entering a formula into a spreadsheet)
Strategic Principle - task performed by adapting guidelines (e. g., doing a financial projection in a spreadsheet)

Bloom's Revised Taxonomy

Design Architecture Matrix

Putting the above two concepts into a matrix gives us an idea of what type of activities the learners need in order for them to learn the required performance skills. The chart below lists various activities and aids that can help increase the possibility of turning learning into performance. Since the chart will be cut off in the blog, this link will bring up the chart in a new window: eLearning Design Chart.

	Fact	Concept	Process	Procedure	Strategic Principle
Remembering (or being able to locate data by searching)	EPSS or mLearning for finding facts Multiple choice, puzzles, or Drag and Drop for learning facts	EPSS or mLearning for finding examples Demonstration Reading or podcast Social Learning Media - learning from others	EPSS or mLearning for finding the activities Social Learning Media - learning from others Demonstration (rich media) Reading or podcast Multiple choice, puzzles,or Drag and Drop for learning the events	EPSS or mLearning for finding the steps Social Learning Media - learning from others Demonstration (rich media) Reading or podcast Multiple choice, puzzles, or Drag and Drop for learning the steps	EPSS or mLearning for discovering the basic principles Social Learning Media for discussing principles Demonstration (rich media) Reading or podcast Multiple choice, puzzles, or Drag and Drop for learning the principles
Understanding		Matched example/non-example pairs Demonstration Reading with simple graphics	elearning, EPSS, or mLearning for demonstration (rich media) Social Learning Media - discussing and sharing Demonstration Reading with graphics Podcasts	elearning, EPSS, or mLearning for demonstration (rich media) Social Learning Media - discussing and sharing Demonstration Reading with graphics Podcasts	Social Learning Media - discussing and sharing eLearning - Interactive Scenario Case study followed by questions
Applying		Case study followed by questions Drag and Drop or puzzles Social Learning Media - sharing experiences Creating wiki entry or blog post	EPSS - list activities eLearning - Interactive Scenario Social Learning Media - sharing and receiving guidance Creating wiki entry or blog post	EPSS - list steps eLearning - Interactive Scenario Social Learning Media - sharing and receiving guidance Creating wiki entry or blog post	EPSS - list activities Social Learning Media - sharing and receiving guidance Creating wiki entry or blog post eLearning - Interactive Scenario
Analyzing		eLearning - Interactive Scenario Social Learning Media - reflecting and sharing	Matched example/non-example pairs Social Learning Media - reflecting and sharing	Matched example/non-example pairs Social Learning Media - reflecting and sharing	Case study followed by questions Social Learning Media - reflecting and sharing
Evaluating		Social Learning Media - sharing experiences and creating blog posts	Case study followed by questions Social Learning Media - sharing experiences and creating blog posts eLearning - Interactive Scenario	Case study followed by questions Social Learning Media - sharing experiences and creating blog posts eLearning - Interactive Scenario	Social Learning Media - sharing experiences Case study followed by questions and blog post or wiki entry
Creating		Social Learning Media - project interaction (chat, wiki, blog)	Blended Learning - elearning and face-to-face - Action Learning Social Learning Media - project interaction (chat, wiki, blog)	Blended Learning - elearning and face-to-face - Action Learning Social Learning Media - project interaction (chat, wiki, blog)	Blended Learning - elearning and face-to-face - Action Learning Social Learning Media - project interaction (chat, wiki, blog)

Fact

Concept

Process

Procedure

Strategic Principle

Remembering
(or being able to locate data by searching)

EPSS or mLearning for finding facts

Multiple choice, puzzles, or Drag and Drop for learning facts

EPSS or mLearning for finding examples

Demonstration

Reading or podcast

Social Learning Media - learning from others

EPSS or mLearning for finding the activities

Social Learning Media - learning from others

Demonstration (rich media)

Reading or podcast

Multiple choice, puzzles,or Drag and Drop for learning the events

EPSS or mLearning for finding the steps

Social Learning Media - learning from others

Demonstration (rich media)

Reading or podcast

Multiple choice, puzzles, or Drag and Drop for learning the steps

EPSS or mLearning for discovering the basic principles

Social Learning Media for discussing principles

Demonstration (rich media)

Reading or podcast

Multiple choice, puzzles, or Drag and Drop for learning the principles

Understanding

Matched example/non-example pairs

Demonstration

Reading with simple graphics

elearning, EPSS, or mLearning for demonstration (rich media)

Social Learning Media - discussing and sharing

Demonstration

Reading with graphics

Podcasts

elearning, EPSS, or mLearning for demonstration (rich media)

Social Learning Media - discussing and sharing

Demonstration

Reading with graphics

Podcasts

Social Learning Media - discussing and sharing

eLearning - Interactive Scenario

Case study followed by questions

Applying

Case study followed by questions

Drag and Drop or puzzles

Social Learning Media - sharing experiences

Creating wiki entry or blog post

EPSS - list activities

eLearning - Interactive Scenario

Social Learning Media - sharing and receiving guidance

Creating wiki entry or blog post

EPSS - list steps

eLearning - Interactive Scenario

Social Learning Media - sharing and receiving guidance

Creating wiki entry or blog post

EPSS - list activities

Social Learning Media - sharing and receiving guidance

Creating wiki entry or blog post

eLearning - Interactive Scenario

Analyzing

eLearning - Interactive Scenario

Social Learning Media - reflecting and sharing

Matched example/non-example pairs

Social Learning Media - reflecting and sharing

Matched example/non-example pairs

Social Learning Media - reflecting and sharing

Case study followed by questions

Social Learning Media - reflecting and sharing

Evaluating

Social Learning Media - sharing experiences and creating blog posts

Case study followed by questions

Social Learning Media - sharing experiences and creating blog posts

eLearning - Interactive Scenario

Case study followed by questions

Social Learning Media - sharing experiences and creating blog posts

eLearning - Interactive Scenario

Social Learning Media - sharing experiences

Case study followed by questions and blog post or wiki entry

Creating

Social Learning Media - project interaction (chat, wiki, blog)

Blended Learning - elearning and face-to-face - Action Learning

Social Learning Media - project interaction (chat, wiki, blog)

Blended Learning - elearning and face-to-face - Action Learning

Social Learning Media - project interaction (chat, wiki, blog)

Blended Learning - elearning and face-to-face - Action Learning

Social Learning Media - project interaction (chat, wiki, blog)

What tools are you using to to help ensure your learning platform goes beyond a page-turner?

ADDIE Backwards Planning Model

2010-11-22T10:22:00.001-08:00

I have been working on this model for some time, so I wanted to present my latest version.

The ADDIE Backwards Model is quite similar to most other ADDIE type models. Note that the Learning Platform (Implement) rests on the Analysis, Design, Development, and Evaluation Phases. The steps in the Analysis Phase closely align with Phillips' Needs Model and Kirkpatrick's Four Levels of Evaluations. In addition, the Analysis steps align with the Design and Development steps:

Click for a larger version with “clickable” links

Analysis Phase

Business Needs - how a learning initiative will support the organization's initiatives, strategies, or goals
Job Performance Needs - determine the cause of the performance deficiency that is preventing the business unit from reaching its objectives and identify the performance required to reach it
Training Needs - define appropriate performance, instructional, and informational material (includes both formal and informal)
Individual Needs - ensure the goals and tasks will be judged by the learners as important and doable

Design Phase

Develop Objectives - what tasks the learners will be able to perform after they finish the learning process
Develop Tests - how well the tasks must be performed
Identify Learning Steps - how to perform the tasks
List Entry Behaviors - what the learners must know before entering the learning process
Sequence - sequenced and structured to provide the best opportunity for learning that will lead to performance

Development Phase

List Learner Activities — activities that help the learners perform in order to meet the Business Needs.
Choose Delivery System — the medium is selected that will not only best deliver the learning platform to the learners, but also has the least interruption on their jobs (performance aids, social media, informal learning techniques, etc.)
Review Existing Material — see if any preexisting content can be recycled to meet the performance needs
Develop Instruction — the courseware, such as the activities, performance aids, content, context, etc. are created
Synthesize — combine into a coherent whole so that it best integrates the information and activities into a learning platform that fosters performance
Validate Instruction — ensure the learning platform helps the learners to reach the business objective and informs them of the need to perform to the required standards

Implementation Phase

The Analysis, Design, and Development phases provided the underlying support to ensure the learning platform:

Performs as predicted (solves a real business need)
Flows with the job performance needs (improves job performance rather than interrupt it)
Supports all training and learning needs
Ensure the learners see the training as important and doable so that they are motivated to engage in it

Evaluation Phase

Aligning the ADDIE model with Kirkpatrick's Four Levels of Evaluation helps to ensure the learning platforms performs as expected

Ideas Favor the Connected Mind

2010-11-11T07:21:00.000-08:00

In a recent TED Talk, Steven Johnson, the best-selling author of six books on the intersection of science, technology and personal experience, gives a presentation titled, Where good ideas come from.

Connecting

He notes how England's coffee houses became a social meeting place for people that began one of the great intellectual periods in the last 500 years — “The Enlightenment” — in that it allowed different people with different backgrounds and different fields of expertise a place to meet and exchange ideas.

An idea is a network (a new configuration); however, that idea is normally cobbled together from whatever parts happen to be nearby — we take ideas from other people, people we learn from, people we meet in the coffee shop, and then stitch them together into new forms. Fred Stratton (CEO of Briggs & Stratton) once said that genius lay in the ability to see how two or more ideas that nobody else sees as related are indeed related. This ability to make an analogy between different ideas unlocks a world of potential. And the means that we get to see various ideas is often accomplished by connecting with others.

These meeting places where we connect are often called “common environments.” However, social media tools, such as blogs, micro blogs (e.g., Twitter & Yammer), file sharing (e.g., Flickr & SlideShare), virtual meeting places, (e.g., Adobe Connect & Elluminate), social sites (e.g. Facebook & MySpace), and wikis now provide a virtual bridge by acting as the common environment in many instances. This virtual bridge allows people to interact with each other in much the same manner as they would in a common environment, thus they are virtually able to observe, gather new ideas, and learn from others.

Space has shifted as people do not have to be in the same physical location. The availability of the common environment has been greatly extended.

The Formation of New Ideas

While people often say they get their ideas in sort of an eureka minute, they are for the most part unreliable when performing this self-reporting on where they get their new ideas. Rather than being an eureka minute, ideals normally happen when people get together so that they can bounce ideas off other people, absorb the thoughts of others, and then build relationships between the various ideas..

Steven Johnson tells the story of how Charles Darwin wrote that he came up with the idea of natural selection in an eureka minute; however, his notebooks tell a slightly different story — he had the full concept of natural selection in them months before he was actually able to put the final theory forward.

While we might think that our ideas come in one spark, they normally are created in an incubation period consisting of 1) connecting with others 2), seeing a relationship between different ideas, 3) developing each part of the new idea, and 4) the joining of the parts in order to create the finalized idea (it is this last step that gives the illusion that the idea is an eureka moment).

Asking if Something Can be Done

In his talk, Steven Johnson tells the story of when Sputnik was launched, two guys named Guier and Weiffenbach started listening to the pings coming from its signal. After a while they noticed small frequency variations that allowed them to calculate the speed of the satellite. They began talking to their colleagues who had other kind of specialties and about three or four weeks later they mapped the exact trajectory of this satellite orbiting earth.

A couple weeks later their boss, Frank McClure asked them, “You've figured out an unknown location of a satellite orbiting the planet from a known location on the ground. Could you go the other way? Could you figure out an unknown location on the ground, if you knew the location of the satellite?” After thinking about it they discovered it would actually be easier. Guier and Weiffenbach's boss needed to be able to do this as he was working on new nuclear submarines in which it was extremely difficult to calculate the aim of a missile so that it lands on top of Moscow, if you don't know the exact location of a submarine located in the middle of the ocean.

Open Innovation

Guier, Weiffenbach, and McClure opened the avenue of GPS. Thirty years later, Ronald Reagan opened it up and made it an open platform that anybody could build upon and anybody could build new technology that would create and innovate on top of this open platform. A closed system connects with a small number of minds, while an open system has an opportunity to connect with a large number of minds that in turn, greatly increases its chance of becoming a new idea, which in turn can become a new innovation.

The Process of Creating Ideas

Thus, connecting people allows the exchange of ideas that form new ideas, which in turn can create another idea that works best when it is opened up to innovation:

This process of creating ideas is important when designing learning environments.

Agile Design

For example, one of the concepts of Agile Design is bringing the designers, managers, learners, and subject matter experts and/or exemplary performers in on the planning stage as a high degree of collaboration needs to take place to accurately identify the problem and solution. When extending instructional system design to solve complex problems, you need to fully immerse yourself in the problem to fully understand it.

Extending Instructional Design

To accomplish either of these requires connections so that the idea creation process can begin.

And we need to not only include the process of creating ideas in our building of learning platforms, but also extending them to the learners so that they can create ideals that will help lead to innovation. Rather than build walls, have no walls. The physicist Freeman Dyson once wrote:

When writing, I close the door, but when doing science, I leave it open. Up to a point you welcome being interrupted because it is only by interacting with other people that you get anything interesting done.

What have you done lately to help increase the creation of ideas within your organization — have you helped to tear down the walls rather than build walls?

Blog Book Tour: Social Media for Trainers

2010-09-20T07:58:00.000-07:00

Welcome to the seventh stop of the Blog Book Tour for Jane Bozarth's new book, Social Media for Trainers. If you have been following the blog book tour, then you know Jane's book not only provides an introduction for understanding how to use social media tools, such as blogs, Facebook, Twitter, and wikis; but is also chock-full of examples. And these examples are quite important as they provide models for learning.

However, perhaps you are still somewhat leery of using social media for learning, thus I'm listing a few points for you to consider.

We often think of new technologies or concepts as being silver-bullets or replacements for present training and learning initiatives. Past examples include video tapes and elearning. And yes, sometimes they are able to stand by themselves as the learning platform; however, they often work much better in blended solutions. Thus, social media should not normally be thought of as a stand-alone solution, but rather as extensions of face-to-face exchanges in order to extend the learners' networks.

Social media can provide a virtual bridge by acting as the common learning environment (see Instructional Design — Social Learning and Social Media). Thus it extends and in some instances may replace the required social interactions that takes place in a lot of learning processes.

As noted earlier, examples provide learners with real models. Social learning works in a similar manner in that it allows the learners to perceive others for comparison and self-evaluation. In addition, we can bounce ideas off of them and are often a neutral source of information, which may help or speed several forms of instrumental learning (Conte, Paolucci, 2001). We know these social interactions are important because while we might picture someone learning informally as being a “lone learner,” studies have shown that during an informal learning episode a learner normally interacts with an average of 10 people (Tough, 1999). And even though you might only be interested in formal learning, you have to support these informal learning episodes because informal and formal learning are closely tied together — an average of one-hour of formal learning spills over to four-hours of informal learning (Cofer, 2000).

Thus to transform training from an event to a real process you have to support the informal learning that accompanies the majority of formal learning. Bell (1977) used the metaphor of brick and mortar to describe the relationship of formal and informal learning. Formal learning acts as bricks fused into the emerging bridge of personal growth. Informal learning acts as the mortar, facilitating the acceptance and development of the formal learning. He also noted that informal learning is not a replacement for formal learning processes as it is this synergy that produces effective growth.

To make training an effective process, you really have think about tying the various parts of learning into a whole, such as formal and informal learning, and the social learning that normally needs to accompany the other two. While the main reason for training failing in the past was most likely the failure to link formal learning to a real business need, I would think with all the emphasis on it the last few years that we have now gone beyond that and we now need to refine our efforts. Jane's book can be an important part of the solution.

Next stop for the Social Media for Trainers Blog Book Tour is Gina Schreck, that is scheduled on September 22.

You can follow Jane on Twitter: @janebozarth and @SoMe4Trainers; connect with her Facebook pages: Jane Bozarth Bozarthzone and Social Media for Trainers; or read some of her thoughts and ideas on her blog.

References

Bell, C. R. Informal Learning in Organizations. Personnel Journal, 56, no. 6 (June 1977): 280-283, 313. (EJ 160 136).

Cofer, D. (2000). Informal Workplace Learning. Practice Application Brief. NO 10. U.S. Department of Education: Clearinghouse on Adult, Career, and Vocational Education.

Conte, R. & Paolucci, M. (2001). Intelligent Social Learning. Journal of Artificial Societies and Social Simulation. vol. 4, no. 1.

Tough, A. (1999). Reflections on the study of adult learning. Paper presented at the 3rd New Approaches to Lifelong Learning (NALL) Conference, University of Toronto: Ontario Institute for Studies in Education, Toronto, Canada. Retrieved January 8, 2008 from http://www.oise.utoronto.ca/depts/sese/csew/nall/res/08reflections.pdf

On The Media

2010-09-09T10:10:00.000-07:00

A recent On The Media Podcast has several short stories about media that might be of interest to many of you. Unlike their original podcast in which all the stories are combined into one podcast, their web page on the podcast has the stories broken down into short segments so that you can pick and choose the ones you want to listen to.

Field Guide

The first story that might be of interest is titled, Field Guide (transcript), that discusses the popularity of the Madden NFL video game among pros, college, and high school football players. One thing is for certain — the new players are much more familiar with offensive formations, defensive formations, play calls, etc. than new players 20 years ago; and both players and coaches agree that they are learning a lot about football through the video game.

The Uncanny Valley

The next one is titled the The Uncanny Valley (transcript). The story is about simulations and the human sims within them. What is interesting is that if you create a sim that is 95 percent lifelike, it is great; however, if you made one that is 96 percent lifelike, it is a disaster because it is like a a human being with something wrong. That is, if it gets too life-like, but something is slightly off, it starts to look horrifyingly dead and dull — which freaks people out.

The Facebook Effect

Why does Facebook trump all it's competitors? The Facebook Effect (transcript) story says it is because:

You use your real name on Facebook, and while this seems so familiar now, it was really quite revolutionary when Facebook launched
The service improved from a technology point of view
They live by the rule “only the paranoid survive”

The Death of the Web?

While the recent story in Wired magazine left many people scratching their heads, the story, The Death of the Web? (transcript) notes that we have to realize that the “web” is NOT the “internet.” The web is what we surf via our Web browser (web sites and HTML). While the internet is composed of the network, the wires, the routers and all that on which digital information flows. And it is non-web applications that are starting to have a big impact in how we receive our information (think iPad and iPhone).

Tools for Analysis

2010-08-31T11:09:00.000-07:00

A Learner's Framework — While many analysis methods or tools study the task, this one looks at the needs and behavior of the learners and how they view a particular task.

Analysis Template Book (RTF) — A collection of instruments for performing analysis

Affinity Diagrams — A method for identifying connections among issues and to reveal opportunities for innovation. For examples, see this Google search, for instructions, see this PDF file.

Concept Mapping — A diagram showing the relationships among concepts or ideas. For examples, see this Google search, for instructions, see this site.

Flow Analysis — Identifies bottlenecks and opportunities for functional alternatives. For examples, see this Google search, using Word and Powerpoint to draw flow charts. This method uses Excel.

Various Approaches for Analyzing Tasks and Needs — In a traditional needs analysis, the analyst generates a list of tasks to be performed. This list of tools is designed to be completed by job incumbents, subject matter experts and/or supervisory personnel.

What tools do you have in your toolbox?

The ADDIE Backwards Planning Model (updated)

2010-08-06T07:28:00.000-07:00

I have updated and made the ADDIE Backwards Planning Model interactive:

Click the ADDIE Model to go to the site

I would be interested in your thoughts.

The ADDIE Backwards Planning Model

2010-08-04T10:23:00.000-07:00

I have been reorganizing my ISD site and have come up with this ADDIE model:

Click for a larger image

I would be interested in your thoughts.

We Need Real Examples with Social Learning

2010-07-29T12:38:00.000-07:00

The latest podcast from Xyleme Voices, George Siemens on Social Learning Networks: From Theory to Practice is quite informative and thought-provoking; however, it still seems to leave a lot of questions unanswered.

For example, George cites the problem that Apple had with the iPhone's antenna and says it was caused by a people not being connected properly and somehow makes the leap of faith that if the people who knew about it were connected to the others on the design team, such as with social media, then the problem would have been prevented. Yet the problem is much more complex. Any good engineer will tell you that all cellphones that have the antenna inside of them, rather than sticking out, have reception problems because when you put your hand around it, it blocks the signal. But people don't want the antenna sticking out — they want it inside the phone. For example, Nokia jumped in and said their cellphones do not have this problem, yet we have a new Nokia phone in our household and the manual shows you how NOT to hold the phone so you don't block the signal.

Secondly, people want their phones as small as possible, which limits where you can put the antenna. Complicating the problem is that most of the testing was probably done on Apple's campus that gets a good signal as both ATT and Verizon have towers on it; and the problem only occurs in weak reception areas.

Thus we don't know if Apple did not know about the problem or if they knew about it, but decided it was not that big of a problem. The real problem was the way Apple handled the problem as I believe, as some other industry experts, that it was mainly blown out of proportion... after all, we love to jump on big companies.

The real problem is more of the way their public relations people handled the problem. Now could being connected with social media help them perform better, and if so how? That might have been a better and more realistic approach.

The most thought provoking part of the podcast comes about 15:10 minutes into it in which George discusses the need to do a better job of helping the learners learn how to learn (metalearning) rather than just instructing them into what to learn. I also believe that we need to do a better job at this, but we need more realistic examples — the Apple's antenna example just does not cut it.

For example, if I'm designing a course in which the learners need to learn a process to deliver our goods or service, how do I fit social learning and social media in with it? I can see how they help with informal part, but the end of the podcast wants us to a better job with metalearning in our daily training, and this is where we need some real examples.

Social Learning and Social Media

2010-07-14T12:02:00.000-07:00

I recently wrote about bridging the common environment in Social Learning with Social Media tools — Social Learning and Social Media. I would be interested in your thoughts.

Tweet This

15 Years on the Web

2010-07-13T08:00:00.000-07:00

This week marks the fifteenth anniversary of Big Dog being on the web! In late 1994 I subscribed to Prodigy as it was one of the first of the early-generation dialup services to offer full access to the World Wide Web. However, a few months later I decided that I did not want to just surf the web, but actually be a part of it. So I then subscribed to NWLink.com and on July 13, 1995, Big Dog went live.

As you can see, my early design skills were somewhat lacking. I did most of my html coding on MS Notepad and except for a few clipart pictures, the drawings were done on MS Paintbrush — really hi-tech stuff.

The main attraction was the SAT or ISD (ADDIE) page. The April 1996 issue of Training Magazine called my site, "The World Wide Web at its best" (April 1996, p.101). Of course it did not take a whole lot to satisfy people back in those early Internet days!

In early 1997, the Internet Archive's WayBackMachine started archiving my site. Later that summer I was running out of things to write about so I decided to put my 22 years in the Army to good use and created the leadership page, which is now one of the most popular sections of Big Dog.

One of the questions I'm often asked is, “Why Big Dog?” While serving with the 3d Infantry Division, our company had a saying, “To the Big Dog!”, which basically meant, “We can do it!” The saying was based on a bulldog mascot that was designed by Walt Disney Productions (for the cost of one dollar) as the division commander wanted a visual picture of the song, “The Dog Face Soldier” (clink link to hear the song and learn more about it). The saying meant a lot to me so I used Big Dog as part of my site's name.

In the summer of 1998 I received an email from Big Dogs clothing claiming I was infringing on their trademark and ordered that I remove the name... so I replied to them and told them I had no intention to stop using the name... and of course I am still using Big Dog. My daughter, being the jokester, gave me a couple of Big Dog shirts the following Christmas.

About 8 months later I renamed it Big Dog's Bowl of Biscuits, as my dog, Lazy (who went to the big dog house in Heaven), loved dog biscuits, and a few years ago to Big Dog and Little Dog's Performance Juxtaposition as my small dog (Buddy, a mix of chihuahua and dachshund) wanted to be part of the marquee, along with my Big Dog — Rico, whom I adapted from PAWS.

A few years back I was starting to run out of server space and exceeding the monthly quota for people visiting/downloading my site, so I got knowledgejump.com from my service provider — hopefully I'm set for a while.

Rethinking Gagne's Nine Steps of Instructional Design

2010-06-28T08:50:00.000-07:00

Using Gagnè's Nine Steps of Instructional Design as a guide, I created this slideshow to show how it can be used from a constructivist view-point to create a more learner-centered learning environment.

Rethinking Gagnes Nine Steps of Instructional Design

View more presentations from Donald Clark.

Tweet This

Instructional Design Toolkit

2010-06-16T09:52:00.000-07:00

Deciding I need to tie four instructional design models together (ADDIE, Agile Design, Learning Design and an Extension for ISD), I created this brief description - A Framework for Designing Learning Environments.

Instructional Design Framework

2010-06-07T20:25:00.000-07:00

I added a new section to my Instructional Design site — a Framework to aid in the process for creating learning environments. It is located at Instructional Design Framework. It contains two excel workbooks to aid in the design process. Let me now what you think of it and perhaps what sort of framework you use in your design and development efforts.

Outlook for the Learning and Training Profession

2010-05-25T10:37:00.000-07:00

There is an interesting infographic showing that Education/Training Consultants is one of the Best Jobs in America. It is based on the Bureau of Labor's Statistics Occupational Outlook Handbook, 2010-11 Edition. In fact, if you go to the Bureau of Labor's page on Human Resources, Training, and Labor Relations Managers and Specialists, it reads in part:

Employment is expected to grow much faster than the average for all human resources, training, and labor relations managers and specialists occupations. College graduates and those who have earned certification should have the best job opportunities. Overall employment is projected to grow by 22 percent between 2008 and 2018, much faster than the average for all occupations.

Some of the reasons listed for the projected growth of the learning/training profession includes:

Workers need to be trained on new legislation and court rulings that revise standards
Employers are expected to devote greater resources to job-specific training programs in response to the increasing complexity of many jobs and technological advances
As highly trained and skilled baby boomers retire, there should be strong demand for training and development specialists to impart needed skills to their replacements

But as we all know, the learning/training profession lost a number of jobs due to the recent economic woes — the report even indirectly acknowledges this:

Like other workers, employment of human resources, training, and labor relations managers and specialists, particularly in larger companies, may be adversely affected by corporate downsizing, restructuring, and mergers; however, as companies once again expand operations, additional workers may be needed to manage company growth.

Yet, it seems we sometimes take the biggest hits and are often the last to be called back to the workforce. Why? The biggest reason is that we often fail to directly address problems that impact the organization's performance. Managers in line departments want people who can solve some of their problems. In most cases they are not worried about ROIs or other fancy formulas — they simply want the problem to go away. Every problem that disappears is one less task they have to spend time on. Yet when given some of the simplest training problems we often go astray and rather than delivering a solution, we deliver some cute learning program that offers no real impact.

What are you doing today that shows your real worth to the organization?

The U.S. Army, Frog Design, & ADDIE: On Design

2010-05-04T11:03:00.000-07:00

This is the sixth post in a series on extending ADDIE in order to solve wicked or complex problems:

I have been doing a series of posts on Design and how ADDIE relates to Frog Design's Immersion, Convergence, Divergence, and Adaptation (ICDA) when Simon Bostock (@BFchirpy) tweeted about the U.S. Army and Design — Design Thinking Comes to the U.S. Army. And what is quite interesting is their five fundamentals of design as outlined in their Field Manual (pdf):

Apply critical thinking.
Understand the operational environment.
Solve the right problem.
Adapt to dynamic conditions.
Achieve the designated goals.

What makes it interesting is just like Frog Design's ICDA and ADDIE complimenting each other, the Army's fundamentals of design also fits nicely in:

Click the image for a larger chart

ADDIE is probably the one basic tool that every Learning Designer needs to know and understand because 1) it is a good tool for getting new Learning Designers up and running, 2) no matter what planning and design tool you use to create your learning platforms, you will probably always fall back on some of the steps within ADDIE, and 3) it keeps teams of designers in the loop. However, like any good craftsperson, ADDIE should not be the ONLY tool within your toolkit.

Which has been one of the problems within the Learning Design profession for some time — while there are plenty of Instructional Design tools that can replace parts of ADDIE, there has been few or no Instructional “System” Design tools that make worthy replacements. Frog Design's Solving for Problem X framework with ICDA and the Army's Design Thinking seem to be viable options for extending ADDIE when you are faced with a wicked or complex problem because rather than being composed of a series of steps, which do not work well when facing such problems; they take more adaptive approaches. Thus rather than being a guide for solving problems, ICDA is a tool for innovating, while the Army's Design Thinking is for more complex and wicked problems than ADDIE is normally called for.

In addition, in some organizations innovation has become the end, rather than the means; which means we can have a surplus of innovation. For example, in our own profession we recently had elearning, blended learning, distributed Learning, informal learning, social media, and social learning enter our radar screens, which we have almost treated as ends rather than means for actually accomplishing goals within the organization. Thus, as Adam Richardson writes in his book, Innovation X, our job when working with the organization is “not so much to help its people come up with new innovations as to filter, prioritize, and refine the ones they already have.” Thus our goal is now just as much innovation effectiveness, rather than trying to create more innovations.

When trying to make our innovations more effective, we have to think systems rather than a course or content. For example, the iPod became a great product not because it was a great piece of hardware, but rather because it was a great solution to managing digital music and recordings such as podcasts. And this has been our problem for some time — rather than thinking of the end goal, which in our case is often performance, we often only think in terms such as courseware or learning.

Extending ADDIE with the lessons of Frog Design and the U.S. Army should be of great help as we move on to more complex problems.

Evaluate and Adaptation

2010-04-28T11:23:00.000-07:00

This is the fifth post in a series on extending ADDIE in order to solve wicked or complex problems:

Evaluation in ADDIE is normally composed of two parts:

Formative Evaluations: a method of judging the worth of a program while the program activities are forming in order to make on-the-spot corrections.
Summative Evaluations: a method of judging the worth of a program at the end of its activities (summation), with the focus being on the outcome.

In addition, perhaps the most popular methodology for evaluations is Donald Kirkpatrick's Four Level Evaluation Model.

Frog Design's Adaptation is less of a formal approach and more of a causal approach — "stepping back and looking for new directions to go." Wicked or complex problems often have no clear directions when it comes to improving the initial solution, thus you simply take another crack at it to see if you can add new or better functionalities. Evaluation and Adaptation could be scaled as:

The point that you wound up on the first continuum of Analysis and Immersion will more than likely determine the point that you should be on this continuum. That is, if you are working on a simple to complex problem, then you will normally use an Evaluation technique to check your initial solution (left side of continuum). However, if you are working on a complicated to complex problem (wicked problem), then you will normally be to the right of the scale and thus use Adaptation to check your solution.

Next up, final wrap-up on innovating with ADDIE by using Immersion, Convergence, Divergence, and Adaptation.

Development and Divergence

2010-04-27T06:18:00.000-07:00

This is the fourth post in a series on extending ADDIE in order to solve wicked or complex problems:

Development in ADDIE is typically thought of as creating the learning content, products, and services, in addition to selecting the media that will carry or deliver them to the learners. Frog Design defines Divergence as branching out beyond what is normally does in its ecosystem in order to take in a wider footprint that provides a holistic solution:

The Development and Divergence Continuum works differently from both the other two ones:

In the Analysis and Immersion Continuum the complexity of the environment determines at what point you are on the scale.
In the Design and Convergence Continuum the goal is to aim for the center of the scale.

While the goal in the Development and Divergence Continuum is to aim for both ends — combine “what works” with “what will make it better."

For example, blended learning is normally considered a combination of elearning and classroom learning. eLearning allows the learners to learn at their own pace, while the classroom portion provides the needed social engagement. The Army has found this to be a superior form of learning and is now moving to a dL (distributed Learning) environment. Note that the “d” is not capitalized as only the “Learning” is emphasized — classroom learning is used when it makes sense.

Another example is the greater consideration of informal and social media when creating formal learning platforms.

Thus, rather than just working from just one point in the scale, Development can often be improved by working from two or more points in the scale.

Up next — Evaluate and Adaptation.

Design and Convergence

2010-04-23T08:00:00.001-07:00

This is the third post of a series on extending ADDIE in order to solve wicked or complex problems:

Post 1: Innovating With ADDIE
Post 2: Immersion and Analysis

In the last post I discussed using a mixture of Analysis and Immersion in order to gain insight and understand the shape of the problem. The next step in Frog Design's framework for solving problems is Convergence — the bringing together of everything you learned in the first step in order to integrate things together; or in other words — to create a whole.

Now this is basically what you do with the Design phase of ADDIE, except with ADDIE, it implies a set of steps, such as developing learning objectives, identify the learning steps, developing tests, etc. If we look at the two terms being placed on a continuum it would look like this:

Unlike the Immersion and Analysis Continuum in which the complexity of the environment determines where you would be placed on the scale, the goal here is to aim towards the middle. That is, if you follow the design process too closely, then you might be turning it into a process model as discussed in the ADDIE Timeline that may take all the creativity out of your learning platform; in addition, it may be moving too far away from Agile Learning Design in which 1) individuals and interactions come before processes and 2) you need to respond to change rather than just following a plan.

And if you go too far to the right side of the continuum then you may end up loosing focus of the goals and objectives of the learning platform. Thus, it is a balancing act of determining the correct level of process (design) with the the right of amount of creative freedom (convergence). If you find yourself at one of the extreme ends of the continuum then you need to seriously question if this is really the correct level of both design and convergence that is required for the project you are working on.

One method that will help you from straying too far to the right or left is to use a mapping or graphic technique, such as Cathy Moore's Action Mapping:

Design Lively Elearning with Action Mapping

The business of design is to make things that people really want: useful, usable, and desirable — Shelley Evenson, associate professor of interaction design at Carnegie Mellon University's School of Design.

Bill Moggridge (2007) wrote that iterative prototyping, understanding people, and synthesis are the core skills of design and that these skills are central to design:

Iterative Prototyping: successive small-scale tests on variations of a limited function prototype in order to permit continual design refinements.
Understanding People: having a basic foundation of the cognitive and behavioral sciences.
Synthesis: applying prior knowledge and skills to produce a new, innovative, or original whole.

The Design in ADDIE and the Convergence in the Framework for Solving Problems may be thought of as a combination of Shelley Evenson's quote and Bill Moggridge's design fundamentals in order to create a holistic solution.

Up next, the Continuum of Divergence and Development.

Reference

Moggridge, B. (2007). Designing Interactions. Cambridge, Massachusetts: The MIT Press.

The Immersion and Analysis Continuum in Simple to Complex Environments

2010-04-21T08:06:00.000-07:00

In my last post I discussed Innovating With ADDIE by using the X Problem Solving Framework of Immersion, Convergence, Divergence, and Adaptation in conjunction with ADDIE. In this post I'm going to elaborate on the first step of the framework — Immersion and Analysis:

When presented with performance or learning challenges there is a continuum on how to approach the problem that is composed of analysis on one end of the scale and immersion on the other end.

Allison Rossett & Kendra Sheldon defined Analysis as “the study we do in order to figure things out” (Beyond the Podium: Delivering Training and Performance to a Digital World ). This term is normally associated with the traditional steps of analysis, such as performing a needs assessment, task analysis, and building performance measures.

While on the other end of the continuum is immersion. It is defined by Adam Richardson of Frog Design as “the method of soaking yourself in understanding what the problem is that you are facing in addition to informing the other three parts of the framework.” This can be thought of jumping into the problem in order to understand it, but not being able to use traditional analysis tools or instruments due to the complexity of the problem.

In most learning or performance design situations we don't use one or the other but rather a mixture of the two, thus it is a Sliding Scale Continuum:

For example:

Left of Continuum

Simple problems will be composed mostly of analysis because we have Exemplary Performers that we can use for models. While this is normally one of the easier learning platforms to design, it does have a couple of pitfalls. The first is thinking that since it is fairly easy to design, it is also easy to learn and perform, thus we fail to build practice time into the learning platform. Designers often become so absorbed in their work that they fail to realize how much time they are putting into it, thus they spend weeks working on the task, then think they can transform it into a 30 minute information dump. The second pitfall is failing to support the informal learning that must occur after the formal learning because there is an average of a 1:4 ratio in which one hour of formal learning produces four-hours of informal learning.

Middle of Continuum

As we move to more complicated design environments there is no Exemplary Performers to be found, thus we have to rely on our own understanding of the problem and other experts who have knowledge of the type of problem you are trying to solve. Thus we are moving towards the middle of the continuum:

We are doing analysis in that we are interviewing experts to create a best practice.
In addition, we are immersing ourselves into the environment of those most affected by the learning solution in order to complete our understanding of the problem that the experts cannot elaborate on.

Like the simple environment, you will have to watch for the two pitfalls; in addition, in this complicated learning environment you will normally perform more sets of iterations to ensure you get the feedback of those affected.

Right of Continuum

As we move farther to the right on the scale to complex learning design environments there are no Exemplary Performers and few or no experts to draw upon, thus we get most of our information by totally immersing ourselves into the environment of those most affected by the problem in order to “paint a picture of an emergent practice.” This is because as Rittel (1972) discovered — the best experts within these types of environments are those affected by the solution — since they are the only ones to have experienced the complexity of the problem, they are the best experts for helping to improve that environment.

You need to look for solutions that support informal learning. This is because the complexity of such environments normally needs a small seed of formal learning with large nourishments of informal learning.

Performance Model

In addition, a performance model (such as the one shown below and I discussed in this post) acts differently in the continuum of analysis and immersion.

That is, when you are presented with a problem that is mostly on the analysis side of the continuum, the type of performance problem will normally become apparent within a short time and it will be more of a simple nature. However, as the problem starts to move to the immersion side of the continuum, you will find yourself having to dig deeper and longer to discover the true nature of the problem. In addition, rather than being a simple problem, it will normally be composed of several parts that require multiple solutions, such as fixing a process, creating a learning environment, and redesigning work stations.

Thus, when presented with performance problems we have to think along the scale or continuum of Immersion and Analysis, depending upon the complexity of the problem.

In my next post we will discuss the Sliding Scale Continuum of Planning that is composed of Design on one end and Convergence on the other end.

Reference

Rittel, H. (1972). On the planning crisis: Systems analysis of the “first and second generation.” Bedriftsokonomen. No. 8, pp.390-396.

Innovating With ADDIE

2010-04-12T10:02:00.000-07:00

Every few years the theme of “ADDIE must die” pops up on the radar. This would be a good thing except there are always two major flaws in the argument:

Instructional Designers are misinformed about what ADDIE is — they refuse to believe that the development of a model such as ADDIE is also iterative in nature — their arguments are always about the first iteration of ADDIE. It almost seems as if they want to stick with the practice of waterfall methods.
They offer no suitable replacement.

To combat the first flaw I have written several posts on ADDIE. I recently combined these posts into an ADDIE timeline with a discussion on the timeline.

I don't have an answer for the second flaw, although I did write a series of articles on Agile Learning Design that can be used to extend ADDIE.

Which brings me to the purpose of this post — I recently listened to an outstanding podcast, Got X Problems? by Frog Design in which they discuss innovation and why are so few organizations are seeing results. In this episode they discuss solving wicked or complex problems from 21st century challenges that defy conventional planning. What struck me about their method is that it almost perfectly aligns with ADDIE, except that you have to approach it from a slightly different mind-set. In the podcast they use a four-step process:

Immersion — soaking yourself in the problem to harvest customer insights
Convergence — bringing together all things such as physical, technology, software, and services into a logical design
Divergence — exploring new advantages
Adaptation — stay nimble in a fast-moving environment by going in new directions when facing roadblocks — based on learnings

These four mind-sets would map with ADDIE as:

If you need a different mind-set when facing difficult challenges then I encourage you to listen to the podcast. The RSS feed is http://feeds.frogdesign.com/frog-design-blog/design-mind-on-air. The episode is Got X Problems, dated March 15, 2010. The podcast is about 15 minutes long.

Team Leadership Model

2010-04-05T13:33:00.000-07:00

A model for implementing Team Leadership: Team Leadership Model

Informal Learning and Leadership

2010-03-30T14:22:00.000-07:00

I have updated my Informal Learning page. A lot of the updates are from my recent blog postings on informal learning, social media, etc. It seems like a lot of the concepts can be tied to Leadership. I have to think it over (reflect) and see what I can come up with. Perhaps something similar to what I did with AARs

Performance and ADDIE Models

2010-03-29T09:00:00.000-07:00

Clark Quinn has an informative post where he discusses the need for Performance Analysis, Learner Experience Design, and ADDIE. I think I have it somewhat summed up in this chart I created for my web site (note that clicking the chart will bring up a larger image that has clickable links):

Performance Design Concept Map

Click chart for a larger image & a clickable map that will take you to the various parts

Performance

ADDIE and other ISD and ID models were never designed to to discover performance problems, thus when confronted with such problems you need to discover the actual cause. Note the some managers will present every performance problem as a “training” problem, which means you need to ensure the problem is training related or requires some other performance solution.

The Performance Analysis Quadrant (PAQ) is a tool to help in the identification. By discovering the answer to two questions, “Does the employee have adequate job knowledge?” and “does the employee have the proper attitude (desire) to perform the job?” and assigning a numerical rating between 1 and 10 for each answer, will place the employee in 1 of 4 performance quadrants:

Performance Analysis Quadrant (PAQ)

Quadrant A (Motivation): If the employee has sufficient job knowledge but has an improper attitude, this may be classed as motivational problem. The consequences (rewards) of the person's behavior will have to be adjusted. This is not always bad as the employee just might not realize the consequence of his or her actions.
Quadrant B (Resource/Process/Environment): If the employee has both job knowledge and a favorable attitude, but performance is unsatisfactory, then the problem may be out of control of the employee. i.e. lack of resources or time, task needs process improvement, the work station is not ergonomically designed, etc.
Quadrant C (Selection): If the employee lacks both job knowledge and a favorable attitude, that person may be improperly placed in the position. This may imply a problem with employee selection or promotion, and suggest that a transfer or discharge be considered.
Quadrant D (Training and or Coaching): If the employee desires to perform, but lacks the requisite job knowledge or skills, then some type of learning solution is required, such as training or coaching.

Learner Experience Design

Not only must you always use a tool, such as the one above to discover the true solution to a performance problem, ADDIE should almost never be used as a stand-alone solution. Being part of the ISD family, it is very broad in nature, thus it does not go into a lot of details. You need to use it as a plug and play solution — while Clark wrote of Learner Experience Design, you can add any additional components to it on an as-needed, such as Action Mapping, 4C/ID, and Prototyping.

ADDIE

Clark noted in his post that one of ADDIE's failures was being a waterfall method, but as I noted in a past post, ADDIE evolved into a dynamic method in the mid-eighties. ADDIE does make a good checklist; however, use it wisely. If you blindly follow it, then it is nothing more than a process model. However, if you use it in a more creative fashion, then it becomes a true ISD model that enhances the design of the learning process.

ADDIE and the 5 Rules of Zen

2010-03-08T08:03:00.000-08:00

As I noted in my last post, we often go astray with our learning initiatives because we fail to properly orient ourselves to the learning/performance environment. While ADDIE or ISD might look like an overblown, time-wasting toolkit, we need to realize that we do not need every tool within that kit. Every project requires a different set of tools. However, there are a few basic ones that will give you direct intuitive insight into the problem at hand. While there are several Zen rules that govern the aesthetics of Japanese gardens and other art forms in Japan, five of them can be directly applied to ADDIE to aid you in creating an energized, but calm learning platform (inspired by a Presentation Zen post).

Analysis = Shizen (自然)

Shizen means naturalness in that it avoids artificial construction. This means analysis must be sought for and expressed in a plain, simple, and natural manner. To get to its natural roots, describe the business need in terms of the performance desired and where they are now. Everything between their present state and desired performance is the performance gap. This gap must be bridged by identifying the skills and knowledge that enable the learners to perform.

Shizen also applies to the learners. Learning becomes difficult when we have to learn new concepts because we have no relationship to them, thus we construct artificial backgrounds. However, when we relate to a new concept naturally by linking it to an experience that is familiar to us, then we learn much faster and deeper. This is why we need to capture the present performance of the learners so that we can create “Advance Organizers” that will link the learner's new knowledge to this present performance or knowledge level. Learners who are given Advance Organizers at the beginning of a learning process have been shown to increase transfer of training.

Design = Shibui/Shibumi (渋味)

Shibui is coolness and beauty through a clear design and nothing more. Think of design as minimalist that articulates brevity. If you decorate or carry it beyond what it was meant to be, then it becomes gauche rather than deco. Think lean by identify the minimal steps and activities that will enable the learners to master the performance rather than rather than overdoing it by including every possibility. To help transform the learning from an activity to a process and reduce the complexity of training; determine the support, such as tools and performance aids, they will need to enable their performance in the workplace.

Development = Fukinsei (均整)

Fukinsei means asymmetry or irregularity. Controlling balance through the use of irregularity and asymmetry is a central belief in Zen aesthetics. For example, the enso or Zen circle is often incomplete to symbolize the asymmetry. Too often we try so hard to create the perfect learning platform by filling in all the blanks that it fails to draw the learners in — we tend to be drawn in more when we can fill in some of the blanks on our own. Designers also tend to carry it to the extreme by adding too much content — it overwhelms the learners with too much information. Thus we need to strive for some incompleteness, irregularity, and/or asymmetry in order to draw the soon-to-be performers into the learning process.

Fukinsei design also implies a deductive approach. That is, rather than being presented with a complete set of concepts, rules, and strategies, that explicitly instructs the learners, they must explore and experiment with the task to infer and learn the rules, principles, and strategies for effective performance (discovery or experimental learning). This approach has been shown to build “Adaptive Expertise” — becoming more adaptable in order to solve unstructured and ill-defined problems. Also closely related to this is “Error-based Learning” to allow for more control processing.

“Nature itself is full of beauty and harmonious
relationships that are asymmetrical yet balanced.
This is a dynamic beauty that attracts and engages.”
- Garr Reynolds

Implement = Kanso (簡素)

Kanso means simplicity by eliminating the unneeded things (clutter) to find the clear structure. Clear structure does not necessarily mean a linear list, but rather a choice of options. However, when it comes to practicing the skills, ensure it includes whole-task practice that will help to carry performance to the workplace.

A Kanso style choice of options is also critical as providing learner control has been shown to also build “Adaptive Expertise.”

Evaluation = Seijaku (静寂)

Seijaku is tranquility or an energized calm. An effective learning platform should bring a sense of peace to the organization by eliminating the gap between the present performance and the desired performance identified in the analysis phase. The use of iterations will allow you to quickly lower the unintentional noise and disturbance. Continue iterating until the disturbance has been transformed into the desired state of tranquility.

Seijaku should also be carried one step farther — having the learners participate by monitoring and evaluating in order to help them increase their metacognitive skills.

Picture Credits

deco by Ohad. enso circle by Vibhav. bamboo/rocks and dock by istock. pods and bento by me.

Creating a Learning Platform for Boyd's OODA Loop

2010-03-02T09:27:00.000-08:00

In my last post I talked about getting more out of AARs by incorporating them with informal learning and social media tools and then pushing it with formal learning in order to capture lessons learned. This approach allows you to set up a learning strategy that is a real process rather than a one time event. Similar approaches can be done with the OODA loop.

Although originally developed for fighter pilots and then adapted by the Armed Forces, Boyd's OODA Loop is now used by organizations as a decision-making tool that a person or team cycles through as rapidly as possible to help them to stay ahead of their competitors.

It is composed of four actions (Observe, Orient, Decide, Act).

Observe

In this part of the action loop the observer scans the environment in order to gather information and learn about it. However, in this day of abundant information, think more of applying the right filters to gather the information that will be most useful to you. As Clay Shirky notes, “It's Not Information Overload. It's Filter Failure:”

While we are still far from having the perfect filters, we are at least heading in the right direction. For example, I have a list of some of the resources that I use, to include on-line publications, discussion groups, web sites, and blogs that provide me with information about the learning industry. The ones with RSS feeds provide the most value as they feed into my primary browser (Safari) and its is set up so that I only get a small portion of the content. Thus I can quickly scan the latest feeds and then click on them if I determine they have information that interests me. Organizations can help their workers by setting up master portals of relevant links and feeds for each of their various workgroups, and then using a learning platform to help the workers to select the ones that pertain to them the most so they can build their own personal learning environment (PLE) for scanning their environment.

Another useful tool is the ones that belong in the microblogging category, such as Twitter or Yammer, that not only allows our network to send us relevant information and vice-versa, but also allows us to ask for help when we are trying to find information. This may also require a learning platform that will help the workers to learn to send useful information without flooding their coworkers with useless information.

In addition, organizational blogs allow the workers to easily publish their observations so that others may benefit from them. An added benefit is that the readers can provide comments so that the observer/writer receives feedback as discussed in the next part of the action loop — Orient.

Orient

The second part of the action loop is synthesizing the data and information gathered during "observing" into a mental image of the circumstances. As more information is received, you "deconstruct" old images and "create" new images. Within this part of the action loop you have to question past beliefs to see if they still hold true when compared to the new information.

This is perhaps the hardest part of the four actions because we basically fail to ask the right questions and to question our beliefs. For example, Chris Argyris discovered that even highly educated adults stick with Single-Loop Learning rather than using Double-Loop Learning by questioning underlying assumptions:

Another example is the Army, who is best known for creating teams rather than groups, having to create a special environment (AAR) so that the troops could learn from their mistake rather than playing the blame game.

Boyd also noticed this phenomenon in that he spent most of his time emphasizing orient in the OODA loop.

The learning profession also shows this trait of failing to orient themselves in that they rush through the analysis part of a project so fast they fail to properly align the learning platform with real business needs. Secondly, they fail to scan the environment far enough ahead, so rather than designing learning processes, they end up with a learning event that fails to transfer to the workplace. And finally, they fail to evaluate their original beliefs, thus rather than capturing lessons learned, they are doomed to a cycle of repeating their mistakes. No wonder we always hear the old mantra, “ADDIE must die!” — we would rather rush ahead with ill-designed rapid development programs that simply concentrate on development and delivery, rather than properly orientating ourselves to the present environment.

Not only do we fail to question our beliefs, but when we ask others for feedback, they normally evaluate before trying to fully understand. Carl Rogers discovered five main categories of feedback. They are listed below in the order in which they occur most frequently in daily conversations:

Evaluative: Making a judgment about the worth, goodness, or appropriateness of the other person's statement.
Interpretive: Paraphrasing - attempting to explain what the other person's statement means.
Supportive: Attempting to assist or bolster the other communicator.
Probing: Attempting to gain additional information, continue the discussion, or clarify a point.
Understanding: Attempting to discover completely what the other communicator means by her statements.

Notice that we often make judgments more often than we try to understand.

Thus a formal learning strategy is needed to help the workers develop skills that allow them to question their beliefs and asking probing questions to see if their beliefs are still valid. In addition, this needs to stretch in the other direction so that when others ask for our opinions, we use the five categories of feedback in the proper direction — we gain a level of understanding by using probing, supportive, and interpretive feedback before we provide an evaluation.

Secondly, the microblogging tools mentioned in the Observing portion of the action loop can be used just as effective, if not more, to get the a broader range of opinions from others.

In addition, the use of wikis or other social editing tools, allow teams performing the OODA Loop to rapidly edit group documents as they reorient themselves.

Decide

The use of PLEs with RSS feeds, blogs, microblogging tools, and wikis in the OODA loop allows the workers to consider all the options and select a subsequent course of action.

Act

This is where you carry out the conceived decision. Once the result of the action is observed, you start over. However, before you do, capture lessons learned by using an AAR so that learning becomes part of the natural process:

Note that in combat (or when competing against your competitors), you want to cycle through the four action steps faster and better than the enemy, hence, it is a loop.

Formalizing the Informal: Been there, done that

2010-02-16T09:40:00.000-08:00

@Ecollab asks, “Can we formalize informal learning?” My answer, “We've been there, done that.” Except for perhaps compliance learning programs, formal learning processes are designed to do exactly that. For example:

When we put in place a training platform (learning to do a present job) we are ensuring the workers learn a certain process that should not be left to chance. By chance I mean that the workers MIGHT discover a process that they should be following so that your product and/or service is delivered to the customers correctly, thus they learn it informally. Whenever we initiate a training program we are saying that this process is too important to be left to chance (informal learning).

When we put in place an education program (learning to do a future job) we are ensuring that we have in place a critical position that we cannot do without. Most workers learn to do their coworker's job through informal learning. However, some positions are vital to a critical process, thus we cannot leave it to chance (informal learning).

Whenever we develop workers (helping them to grow so that the organization grows), we are helping them to learn certain skills that would otherwise would have to be left to chance (informal learning). For example, in my last post, Driving the Informal with the Formal, I discussed how to capture lessons learned with an AAR and social media.

Note: If you are lost with the three terms (training, education, and development), see The Three Words Your Customer Must Know.

I think this 80/20 informal/formal thingy is kind of going in the wrong way. We should be spending the majority of our time on 20% of the learning taking place within our organization — remember the Pareto principle? Thus you should be asking:

What processes are critical for delivering our product/service and do we need to ensure that our workers learn them correctly?
What tasks are so vital to a processes that we have to ensure we educate someone to be a backup?
How can we best develop our workers so that we continue to grow as a company? What we think of as the "informal" will most often fall into this category.

The 80/20 rule is exactly what we should be doing; however, ensure that the 20% that you are spending most of your time on is the RIGHT 20%.

Driving the Informal with the Formal

2010-02-10T07:27:00.000-08:00

There has been some discussion recently that as the world gets more complex, formal learning becomes less relevant. However, complex environments imply an "emerging practice," which in turn implies a customary way of operation or performance. Yes, they may be still emerging, but unless these emerging practices are dealt with in a manner that ensures they do indeed become a customary way of doing things, they will soon turn to chaos if left to chance.

For example, I used to work in a manufacturing environment in which our products had a relative firm foothold in our section of the food industry. Mostly because it was new. Thus with little competition we could take our sweet time in how we dealt with growing and moving forward. However, as we become more successful, other companies, both large and small, soon moved into our niche. Thus in order to stay ahead of them we had to produce faster, cheaper, and with more variety; all while still maintaining the quality that our products were known for. Almost every other month we were moving onto something new, such as Lean and Just-in-Time on the manufacturing side and engagement and communication theories on the OD side in order to ward off the increasing intense competition. Basically we would at first embrace the entire concept for a short while and then examine what worked best for us and what had little or negative impacts, and then shed the unworthy ones.

All of this required some sort of formal learning so that we could produce quality products consistently. This process worked similar to Boyd's OODA Loop:

Observe — What best or emerging practice can best help us?
Orient — Test it out to get the proper context.
Decide — What parts work best (shed the mediocre)?
Act — Establish it as a practice (normally with the help of formal learning)

Of course the training we provided had to be created and implemented rapidly and was often done through the use of small chunks. Anything less would only lead to chaos.

The other part of a complex environment is providing better means of informal learning as all this learning taking place would be quite inefficient if placed in a formal learning platform. This means that you must not only support the formal learning though faster and smaller learning chunks, but also support the ever increasing informal learning. One way of accomplishing this is through the use of a three-prong process: 1) capture the Lessons Learned, 2) store it in a social site for easy retrieval, and 3) provide a microblogging tool to help others to learn about recent lessons learned.

AAR

As noted in my last post, informal learning can be just as expensive as formal learning, thus the need to get more bang for your bucks. Part of its inefficiency is capturing lessons learned. That is, while a person or small group may learn something new, that learning is rarely or never passed on to others in the organization. Thus while there may be a lot of informal learning occurring, a lot of it are repeats. One of the best means for combating this is through After Action Reviews (AAR). You can help "push" AARs into the working environment through formal learning, such as:

Classroom — Having the learners discovering which of their projects make good candidate for running through an AAR.
eLearning — Once their projects are nearing their completion, then they receive some just-in-time training for:

Questioning Techniques — Using scenarios, such as, rather than asking, “Robert, why did your equipment operators remove the safety guards from the equipment?,” they select, “Robert, can you tell us about your recent safety inspection?”
Using the four basic steps of an AAR:

Capturing the results of the four steps through mind maps that keep the steps aligned and use a visual mode that allows other learners to more easily understand them.

Wikis

Once the learners have learned to capture Lessons Learned and then actually put it to practice, they need to move them to a storage site, such as a wiki or SharePoint so others may learn from them. Throughout my travels I have noted that some of the most barren social media sites are often corporate wikis. This is because most were implemented in the same manner as the old Knowledge Management systems were — write your knowledge down and then put it in the wiki. Yet, as we soon learned, people don't want to waste their time creating content that someone MIGHT use. But fortunately, using a wiki as a site for Lessons Learned is the perfect vehicle as the content have already been created through the AAR process and what the participants have learned will more than likely be needed by others.

Microblogging

In addition, we need one more tool for carry the Lessons Learned forward — a microblogging tool such as Twitter, Yammer, Present.ly, Socialcast, or Socialwok. While some of the lessons learned may require formal learning techniques, the majority do not. However, simply putting them in a wiki or SharePoint is not enough as people who have done similar projects in the past and are about to start a new project do not normally go digging through sites to see if something new has been learned. Thus such microblogging tools are invaluable for:

People completing a project — they can send a tweet to broadcast the highlights of Lessons Learned that put others in the loop.
People starting a new project — they can announce their new project and ask if there is something has recently been learned about implementing such projects.

Now while some people and indeed, entire groups or teams will naturally take to such microblogging tools, others will not. Thus you have to know your learners — do they need help with learning the new technology? Don't stop with just a class. Ensure you push the learning into the workplace by requiring them to complete x number of tweets per week, or asking x number of questions and tweeting x number of recent learnings.

Hopefully this three step process will be useful in your quest to support informal learning by capturing the Lessons Learned through the use of an AAR, storing it in a social site, such as a wiki or SharePoint for easy retrieval, and then providing a microblogging tool, such or Twitter or Yammer, to help others to learn about recent lessons learned.

How do you plan on supporting the informal?

The True Cost of Informal Learning

2010-02-04T10:54:00.000-08:00

Before I start I want to note that in a very few instances I've been accused of not helping the cause when I question some of the research on informal/formal learning. My position is that I'm neither fer er or agin it, but rather discovering the truth so that we can make logical decisions in our performance efforts. For example, late last year I blogged a post defending informal learning. With that being said...

I'm sure most of you are familiar with this chart in which we are told that 20% of the learning is formal and 70% is informal, yet paradoxical we spend 80% of our training budget on the formal and only 30% on the informal:

Chart 1

The trouble with this chart is that while the percentage of learning is basically correct, the amount we spend on each is based on weak research. Even if all the numbers are correct, it does not matter what the Learning department spends, but rather what the organization spends. In addition, if informal learning was really all that efficient, why does it need it need our support?

One ASTD research project supported by the U.S. Department of Labor reported that the amount of employer investments in workplace training hovers around $210 billion annually and of that, $30 billion is spent on formal training, while the remainder, $180 billion, is spent on informal and on-the-job training (Carnevale, Gainer & Villet, 1990). Thus, informal learning gets 86% of all learning investments, while formal learning programs get 14% of the total investment.

Now if we chart those numbers out, it would look like chart 2 below; however, note that it is NOT correct either:

Chart 2

The reason it is wrong is that expenditure for training/learning departments do not count the learners time (hourly wage) spent in formal training, while informal learning expenditure count the workers hourly wages. To get a more accurate cost we need to add the learner' wages to formal learning's cost. Thus we need to add about 36 billion to the formal learning side, which would give us a chart similar to this:

Chart 3

As we start to get the truer cost of informal learning, we discover that rather than being this highly efficient learning machine, it is probably just about as efficient as formal learning. And if you really think about, it makes sense — just because I'm informally learning does not mean my brain starts working 3 or 4 times faster than when I'm in a formal environment. In addition, formal learning would get the edge if you have a lot of people learning a particular task because because rather than each one of them "building" their own learning package, you have one designer doing the initial legwork for everyone.

Thus rather than being this highly efficient learning machine that we can ignore, it may require just as much of our attention as the formal side of learning.

References

Carnevale, A., Gainer, L., & Villet, J. (1990). Training in America: The Organization and Strategic Role of Training. San Francisco: Jossey-Bass.

The Three Words Your Customer Must Know

2010-01-28T11:01:00.000-08:00

Before I dive into the three words I need to clear up two more misconceptions about training first. Why? Because its one of the words.

Jay includes "training" in his of 8 Dirty Words and gives the following reason, "Training is something you do to someone. Learning is something people do for themselves."

Some of the roughest, toughest people in the training profession are U.S. Marine Drill Sergeants. One of their mottos is that they will not give up un a new recruit as long as he or she does not give up on themselves. Even though their training is some of the best in the world, it is not a one-size-fit-all solution, and they realize they can't "do it" to others; but rather the learners must be willing to learn it themselves. They adjust as long as the learners are willing to learn. I cannot "do it" to you; you must be willing to learn yourself.

Daniel Spira commenting on Jay's post (How Not To Sell Learning / Performance / Training & Development Projects), writes

On the flip side, the term “training” does suggest a strong one-way orientation, as in, “Here is the process. Follow it.” and in that way, appears to presuppose how a given solution will be delivered, which may or may not fit with the client’s needs.

Just because I offer training as one solution, does that mean it is the only solution I am allowed to offer? If I am presented with a performance problem and if it is decided that learning is part of the solution and if the learning meets certain requirements, then yes, training just might be the solution.

OK, on to the three words, Training, Education, Development. These three words are important because they tell the customer when they are going to see a payoff (impact or result) and what must be done to achieve the payoff.

Training

Training is learning that is provided in order to improve performance on the present job. This means training is orientated towards the present. And workplace performance is the only measure to determine its success.

Why must your customer know this? If it is decided that training is the solution, then it has to be carried over to the workplace. It does not require fancy ROI's or other fancy measurements but rather that the agreed upon performance is actually being accomplished once the training is performed. This means there must be an immediate impact once the training has occurred and if this impact is not seen then the training failed. Two quick examples.

I trained forklift operations in manufacturing and distribution centers. Note that these forklifts operate differently than cars in that they have no steering wheel but rather joysticks, their braking system is backwards in that you step on it to make it go and release it to make it stop, etc. Once we have completed the training in a safe place the learners are then certified in the actual workplace. I ascertain that the performance is carried over to the workplace. If I don't ensure the impact occurs then that means I did an half-a** job. Its that simple.

I have also trained SQL on AS/400 systems. The training is important because if they fail to do it correctly they can not only get an incorrect solution but may also bring the system to a crawl. Once we have completed the classroom training, the learners return to the workplace and within a set period of time they are required to build three SQL solutions. Before running each solution they send it to the instructional staff to ensure it is correct. If they fail to complete the assignment their SQL privileges are revoked. We ensure performance carries over to the workplace.

Education

Education in organizations differ from education in schools so don't let the following definition confuse you. Education is training people to do a different job. It is often given to people who have been identified as being promotable, being considered for a new job either lateral or upwards, or to increase their potential. Unlike training, which can be fully evaluated immediately upon the learners returning to work, education can only be evaluated when the learners move on to their future jobs or tasks.

Why must your customer know this term? Since there is a delay between the learning solution and the actual performance, then reinforcement has to occur. If no reinforcement is provided then there is a very good chance that performance will decline (we loose what we do not use). Learning professionals and their customers often get themselves into trouble when they provided training solutions to educational learning problems.

For example, an organization is implementing a new system. Sixty-four days before the new system is to be implemented the learners receive their "training." On the day of the new system implemention things turn into a disaster because the workers forgot what they had learned. The only way a training solution would work is if for example the learners are trained on a weekend and the new system is implemented on Monday. Another method is to provide a performance support solution. If it is decided that the only solution is classroom learning and for some reason the learners will not be able to put their new skills and knowledge to immediate use, then it has to be treated as educational learning, which means reinforcement must be be provided during the delay.

Development

Why must your customer know this term? Most development solutions require a lot of involvement from everyone involved, to include the leaders and the learning specialists. In addition the payoff (impact or result) is normally not going to be seen for some time.

For example, during my talks with my last three customers (all large organizations) I discovered that they all had wikis, but that that they were all basically barren wastelands. And they all implemented them in basically the same way — a training solution was provided. Again, wrong learning solution for the problem. Development solutions that get solved with training solutions is probably the biggest reason that training gets it bad reputation. For example, in my last post I noted that such programs as spirit-building, inculcation of company history and philosophy, and individual growth were identified as training, thus they failed to provide the proper impact when measured against Kirkpatrick's Four-Levels of Evaluation. The reason they failed does not mean they are bad programs, but rather they were treated as training problems rather than development opportunities.

So do yourself and your customers a favor — know the three terms and know when to use them

How Learning Professionals Made Training a Dirty Word

2010-01-27T09:44:00.000-08:00

"Training is what you do to dogs."

Yea. Right. Tell that to the U.S. Armed Forces who used it correctly to become the finest warriors the world has ever seen.

An author writing for a training magazine pens, "How many times have you heard training directors say: "I need to find a way to assure that what I teach in the classroom is effectively used on the job?" "I would estimate that only 10 percent of content which is presented in the classroom is reflected in behavioral change on the job." Shortly after his article appeared, a peer-reviewed journal cites the magazine and article that the "rhetorical question" appears in as "research has shown that only 10% of training transfer to the job." Not too long after that books and speakers are citing the peer-reviewed article (and each other) as proof that training does not work. Learning professionals applause and are awed by this amazing piece of "research."

Yea. Right. If I knew that only 10% of my efforts paid off I would follow Beck's advice, call myself a looser, and go shoot myself.

Two researchers show that Kirkpatrick's Four-Levels of Evaluation is not effective for evaluating "training." However, rather than actually examining "training" programs, they exam such "learning" programs as spirit-building, inculcation of company history or philosophy, and individual growth. And of course the Four-Levels prove ineffective in evaluating these "training" programs. The learning community is happy — the research has not only shown that "training" does not transfer, but also that our methods for evaluating it are ineffective.

Yea. Right. Kirkpatrick never said that his method was valid for evaluating all learning programs, but rather training programs.

The world is becoming so complex that training is no longer a useful concept, besides past "research" has shown that it is pretty much ineffective. It only works with dogs.

Yea. Right. The aerospace industry is perhaps one of the most complex industries around, yet they are able to survive in this complexity through set processes, which of course must be trained.

Great work learning community! We can now forget about "training" and simply do pure "learning" programs. To show that we are still advancing our profession, what word are we going after this decade?

Leading Through a Rear-View Mirror

2010-01-22T07:08:00.000-08:00

Shortly after posting Metalearning and Learning Styles I glanced through the latest edition of Chief Learning Officer and came to an interesting quote, "Many CLOs believe podcasting is oversold because few people are auditory learners and its no different than cassettes or CDs from the past 15 years." You can read the article at, A Look at the Industry in 2010 (the quote is on the last page, top paragraph).

I find this quite interesting as CLOs should be the thought-leaders in our profession and yet they are following Marshall McLuhan's quote spot on:

The past went that-a-way. When faced with a totally new situation, we tend always to to attach ourselves to the objects, to the flavor of the recent past. We look at the present through a rear-view mirror. We march backwards into the future. - The Medium is the Massage.

Rather than looking backwards through a rear-view mirror, they should be looking at McLuhan's tetrad: four laws for looking at our culture and framed as questions:

Enhancement or extend: what improved performance does it provide over the old
Obsolescence: what does it make passe
Retrieval: what new media is pulled from our cultural inventory
Reversal: we tend to over-do the new until we run out of benefits and into detriments

Thus, the four questions that should be asked about podcasting are:

What does the technology extend or enhance?

It extends the benefits of dL (distance Learning) by providing the best medium for delivering lectures. See The effectiveness of m-learning in the form of podcast revision lectures in higher education and Turn on your iPod and learn. Note that lectures are an effective means for departing subject matter knowledge and is just as effective as discussions (Dunkin & Barnes, 1985; Kulik & Kulik, 1979; McKeachie, 1962; Ryan, 1969). In addition, the CLOs in the survey are guilty of placing too much stock in learning styles. The question is not "what media should we use for a learning style," but "what method of learning is best for the knowledge or skill that must be acquired.

Podcasts also enhances the device itself in that it is digital and a more versatile package than cassette or CD players.

What does it make obsolete?

Time spent in the classroom can now be spent on interactions and activities rather than listening.

Podcasting, part of the audio digital file family, is already eliminating the need to keep track of extra media, such as cassettes and CDs.

What is retrieved?

Subject matter knowledge.

What does the technology reverse into if it is over-extended?

It reverts into classroom training that is composed of nothing more than lectures. Thus while podcasts may be used strictly on their own in a few select instances, they should normally be used to train the subject knowledge required for a task, in addition to being used with classroom or elearning activities that will actually give the learners the skills to use the knowledge gained with the podcasts.

Metalearning and Learning Styles

2010-01-20T08:06:00.000-08:00

First, I'm sorry for the long delay between postings but I got tied up with a couple of complex projects. Hopefully future postings will be more timely.

One of the elements in the Periodic Chart of Agile Learning is metalearning — being aware of and taking control of one's own learning (Biggs, 1985). However, since most educational and training activities normally teach learners what to learn, rather than how to learn, this is one task that does not come easily for some learners. One method that can help learners increase their metalearning skills is by giving them an insight into the use of perceptional channels (DeSimone, Werner, Harris, 2002).

Perceptional channels are often referred to as "modalities" — a channel by which human expression can take place and is composed of a combination of perception and memory. It is often used in Learning Style theory. While Learning Styles has been written and blogged about a lot lately, normally in response to two papers, Learning Styles: Concepts and Evidence and Learning Styles and Pedagogy in Post-16 Learning, I believe it is going in the wrong direction. Note that the two papers do NOT disprove learning styles as scientific studies can normally only prove what exists, not what does not exist. For example, in one study of verbal and visual learners, magnetic resonance imaging technology actually shows that the brain processes information differently, depending upon a person's learning style.

Now I'm not advocating that using one's learning style promotes learning, because so far the evidence does not support this, but rather the opposite — the use of one's learning style often hinders or prevents learning. For example, one study found that visual presentation through the use of pictures was advantageous for all adults, irrespective of a high or low learning-style preference for visual images. In addition, it was especially advantageous for those with a strong preference for verbal processing (Constantinidou, Baker, 2002).

This also apples to other types of learning styles. For example, while verbal learners may like to read about something rather than actually try it, they do much better when they actually apply the skills (kinesthetic), rather than reading about it (Matching Teaching Style to Learning Style May Not Help Students).

Rather than teaching learners that knowing and using their personal learning style is best for them, we should be showing them how an over-reliance on one's learning style is actually harmful, thus they need to select the best style or modality for the task on hand. The meta-analysis of educational interventions conducted by (Hattie, Biggs, Purdie, 1996) can help us put metalearning in context with other interventions, which allows us to put our limited resources and efforts into the interventions that have the largest effect sizes:

Intervention	Effect size
Reinforcement	1.13
Student's prior cognitive ability	1.00
Instructional quality	1.04
Direct instruction	0.82
Student's disposition to learn	0.61
Class environment	0.56
Peer tutoring	0.50
Parental involvement	0.46
Teacher style	0.42
Affective attributes of students	0.24
Individualization	0.14
Behavioral objectives	0.12 (interesting?)
Team teaching	0.06

While the above chart does not include metalearning, we can use the research of Marzano (1998) who found that approaches that were directed at metalearning, such as setting goals, choosing appropriate strategies, and monitoring progress are more effective in improving learning outcomes than those which simply aim to engage learners at the level of presenting information for understanding and use. Interventions targeted at improving metalearning produced an average gain of 26 percentile points. While there are a few of ways of calculating effect size and percentile points, I believe 26 percentile points would roughly equal an effect size of 0.80. This would place metalearning in the top five interventions:

Intervention	Effect size
Reinforcement	1.13
Student's prior cognitive ability	1.00
Instructional quality	1.04
Direct instruction	0.82
Metalearning	0.80
Student's disposition to learn	0.61
Class environment	0.56
Peer tutoring	0.50
Parental involvement	0.46
Teacher style	0.42
Affective attributes of students	0.24
Individualization	0.14
Behavioral objectives	0.12 (interesting?)
Team teaching	0.06

In addition, as we begin placing more emphasis on informal learning, rather than formal and nonformal, metalearning's importance would probably rank even higher in the chart. Using learning styles as a tool to help learners learn-to-learn, rather that an instrument for stereotyping them into using a particular style, could help them to target one or more styles to fit a strategy for a particular learning need. If you are interested, here are two free learning style surveys to help you: VAK and the Learning Style Survey.

References

Biggs, J. B. (1985). The role of meta-learning in study process. British Journal of Educational Psychology, 55, 185-212.

Constantinidou, F. and Baker, S. (2002). Stimulus modality and verbal learning performance in normal aging. Brain and Language, 82(3), 296-311.

DeSimone, R.L., Werner, J.M., Harris, D.M. (2002). Human Resource Development. Orlando, FL.: Harcourt, Inc.

Hattie, J., Biggs, J., and Purdie, N. (1996). Effects of Learning Skills Interventions on Student Learning: A Meta-analysis. Review of Educational Research, 66, 99-136.

What's on Your Holiday Playlist?

2009-12-22T12:13:00.000-08:00

My Christmas playlist when I'm not listening to Podcasts:

Song	Group	Album	Time
Happy Christmas (War Is Over)	The Alarm	Starbucks Hi-Fidelity Holiday	3:43
Tchaikovsky: March	Andre Previn (Conducting The Royal Philharmonic Orchestra)	Starbucks Winterludes	2:27
Tchaikovsky: Coffee	Andre Previn (Conducting The Royal Philharmonic Orchestra)	Starbucks Winterludes	3:30
Handel: Hallelujah	Andrew Parrott Conducting The Taverner Players & Choir	Starbucks Winterludes	3:58
Go Tell It On The Mountain	Andy Griffith	Now That's What I Call Christmas! Vol. 2 [Disc 2]	2:21
Christmas Must Be Tonight	The Band	Christmas Party	3:37
Little St. Nick	The Beach Boys	The Spirit of Christmas (Disc 2)	2:00
My Sweet Lord	BeBe Winans	My Christmas Prayer	5:07
Christmas Eve	Billy Eckstine	Have Yourself a Jazzy Little Christmas	3:04
Peace On Earth-Little Drummer Boy	Bing Crosby & David Bowie	The Edge Of Christmas	2:39
Do You Hear What I Hear?	Bobby Vinton	The Spirit of Christmas (Disc 2)	2:55
Rockin' Around the Christmas Tree	Brenda Lee	The Spirit of Christmas (Disc 1)	2:05
The Twelve Days of Christmas	Burl Ives/Percy Faith,Orchestra,Chorus	The Spirit of Christmas (Disc 2)	3:53
Let It Snow! Let It Snow! Let It Snow!	Chicago	Chicago Christmas What's It Gonna Be, Santa?	3:28
Child's Prayer	Chicago	Chicago Christmas What's It Gonna Be, Santa?	3:23
Snoopy's Christmas	The Clumsy Lovers	A Winter's Night	2:50
Frosty The Snowman	Cocteau Twins	Cool Holiday Notes	2:56
Winter Wonderland	Cocteau Twins	Starbucks Hi-Fidelity Holiday	2:53
Blue Christmas	Collective Soul	You Sleigh Me!	3:03
Sleigh Ride	Combustible Edison	Starbucks Hi-Fidelity Holiday	3:10
Gloria In Excelsis Deo	Countdown Orchestra And Singers	All Time Christmas Favorites, Vol 1	1:19
Peace On Earth/Little Drummer Boy	David Bowie & Bing Crosby	Cool Holiday Notes	4:25
Opera Of The Bells	Destiny's Child	Now That's What I Call Christmas! Vol. 2 [Disc 1]	4:37
Sleigh Ride	Diana Krall	Starbucks Holiday: Santa Baby '06	3:26
Christmas (Baby Please Come Home)	Dion	The Spirit of Christmas (Disc 2)	3:47
Silent Night	Dresdner Kreuzchor	Season's Serenade	3:32
Auld Land Syne	The Drifters	A Christmas Party	2:15
White Christmass	The Drifters	The Spirit of Christmas (Disc 1)	2:40
Santa Baby	Eartha Kitt	Starbucks Holiday: Santa Baby '06	3:26
Feliz Navidad	El Vez	Christmas Party	2:34
Have Yourself a Merry Little Christmas	Ella Fitzgerald	Have Yourself a Jazzy Little Christmas	2:56
Rudolph The Red-Nose Reindeer	Ella Fitzgerald	Starbucks Holiday: Santa Baby '06	2:53
Here Comes Santa Claus	Elvis Presley	Starbucks Holiday: Santa Baby '06	1:57
I Believe In Father Christmas	Emerson, Lake & Palmer	Best Of Emerson, Lake & Palmer	3:32
Gloria in excelsis Deo	The English Concert	Season's Serenade	2:23
Silent Night	The Everly Brothers	The Spirit of Christmas (Disc 1)	2:54
A Change At Christmas (Say It Isn't So)	The Flaming Lips	Christmas Party	5:21
The Christmas Waltz	Frank Sinatra	The Sinatra Christmas Album	3:01
The Christmas Song	Frank Sinatra	Starbucks Holiday: Santa Baby '06	3:29
A Charlie Brown Thanksgiving	George Winston	Linus & Lucy - The Music of Vince Guaraldi	2:30
Jingle Bells	Jimmy Smith	Have Yourself a Jazzy Little Christmas	3:13
The Little Drummer Boy	Johnny Cash	The Spirit of Christmas (Disc 2)	2:34
Feliz Navidad [Live]	José Feliciano	Now That's What I Call Christmas! Vol. 2 [Disc 2]	4:56
Joy to the World	Judy Collins	The Spirit of Christmas (Disc 2)	2:33
December Will Be Magic Again	Kate Bush	The Edge Of Christmas	4:51
Christmas Reindeer	The Knife	Christmas Reindeer - Single	5:33
Hallelujah	Leonard Cohen	Starbucks Hi-Fidelity Holiday	4:39
Merry Christmas, Baby	Lou Rawls	The Spirit of Christmas (Disc 1)	2:35
Christmas Is	Lou Rawls	Starbucks Holiday: Santa Baby '06	3:12
Ave Maria	Luciano Pavarotti	The Spirit of Christmas (Disc 1)	5:07
Wade: Adeste Fideles	Luciano Pavarotti; Kurt Herbert Adler: National Philharmonic Orchestra, Wandsworth Boys Choir	Now That's What I Call Christmas! Vol. 2 [Disc 2]	3:32
The Christmas Song	Mel Tormé	Have Yourself a Jazzy Little Christmas	2:48
Caroling, Caroling	Nat King Cole	Cool Holiday Notes	2:03
The Christmas Song (Chestnuts Roasting on an Open Fire)	Nat King Cole	The Spirit of Christmas (Disc 2)	3:10
Please Come Home For Christmas	Pat Benatar	The Edge Of Christmas	3:07
It Doesn't Often Snow At Christmas	Pet Shop Boys	Christmas Party	3:57
Deck The Halls	The Platters	A Christmas Party	2:08
Still, Still, Still	The Plymouth Music Series Ensemble Singers with Philip Brunelle, celeste	Starbucks Winterludes	2:51
2000 Miles	The Pretenders	Christmas Party	3:41
Thank God It's Christmas	Queen	The Edge Of Christmas	4:20
Merry Christmas (I Don't Want To Fight)	The Ramones	The Edge Of Christmas	2:06
Christmas Must Be Tonight	Robbie Robertson	Starbucks Hi-Fidelity Holiday	4:53
We Free Kings	Roland Kirk Quartet	Have Yourself a Jazzy Little Christmas	4:47
Pretty Paper	Roy Orbison	The Spirit of Christmas (Disc 1)	2:46
Little Drummer Boy	The Temptations	Starbucks Hi-Fidelity Holiday	3:26
Thanks For Christmas	Three Wise Men (XTC)	Starbucks Hi-Fidelity Holiday	3:53
Have Yourself A Merry Little Christmas	Tony Bennett	Starbucks Holiday: Santa Baby '06	3:03
Christmas / Sarajevo 12/24 (Instrumental)	Trans-Siberian Orchestra	Christmas Eve and Other Stories	3:25
Christmas (Baby Please Come Home)	U2	A Very Special Christmas, Vol. 1	2:20
Christmas Wrapping	Waitresses	The Edge Of Christmas	5:23
Do You Hear What I Hear?	Whitney Houston	A Very Special Christmas, Vol. 1	3:32

The Periodic Table of Agile Learning

2009-12-17T08:20:00.000-08:00

This is the eighth in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design
Post 3 - Orientation in Agile Learning Design
Post 4 - Designing Agile Learning
Post 5 - Selection in Agile Learning Design
Post 6 - Agile Learning Design: Tools for Learners
Post 7 - Iterations in Agile Learning Design

I was going to close this series out with a mind or concept map of agile learning but after a few attempts I decided that there must be a better way; besides, been there and done that. So I decided to create a Periodic Table of Agile Learning:

click to open

It differs somewhat from the normal periodic table as the main group is the far left column and then the major themes (sub-elements) of each major element is listed in the row next to it. The columns basically keep to the same theme. Thus it is probably more like a matrix than a true periodic table, but I claim poetic licence.

I hope you enjoy it as much as I did creating it! - Don

Iterations in Agile Learning Design

2009-12-10T08:24:00.000-08:00

This is the seventh in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design
Post 3 - Orientation in Agile Learning Design
Post 4 - Designing Agile Learning
Post 5 - Selection in Agile Learning Design
Post 6 - Agile Learning Design: Tools for Learners

In traditional waterfall-type projects, learning platforms are developed in lengthy sequential phases. Learning methods and delivery flaws are normally only discovered during the delivery or evaluation phases. Fixing these defects can waste resources and cause delays to the learning platform or process due to the rework required. This is often referred to as the “1 - 100 - 1,000 rule”: if it cost one to fix it in the initial stages of the project, It will cost 100 times more to fix it at the end of the project and up to 1,000 times more to fix it once it is delivered.

Note: ADDIE or ISD is NOT waterfall (see post on topic), unless the users decide to use the tool in this manner.

Using agile methodologies or concepts allow the designers to test the learning platform up-front in order to ensure it is built upon a sound architecture by discovering the risks and alternatives involved during the planning stage, selecting valid learning objects, and then iterating them in a logical fashion.

Iterations are normally performed using two methods:

Design Iteration (interpretive) — the iteration is performed to test a learning method, function, feature, etc. of the learning platform to a small set of learners to see if it valid.
Release Iteration (statistical) — the iteration is released as a product to the business unit or customer. Although it may not be fully completed or functional, the designers believe that it is "good enough" to be of use to the learners.

A Design Iteration is a micro-technique in that it uses a small set of learners to test part of the learning platform so that you make an interpretation of its effectiveness. This method is normally used for innovative design. A Design Iteration will generally use two types of prototypes:

Drawing or print prototypes — uses paper and pencil models. This allows the design to be quickly sketched out so that you can get input from the learners. It normally solicits more input as the learners do not think the design is "locked-in." In addition it is quite versatile as you can add post-it notes to the paper drawing to simulate drop-down menus, dialog boxes, etc.
Interactive prototypes — uses a more realistic model of the learning platform. Its advantage is that it gets you closer to where you need to be. In addition, the learners think it is more "locked-in," thus once you have captured their basic needs with the drawing prototypes, they are more hesitate to offer suggestions unless there is a real need for the changes (helps to prevent running in circles with design changes).

A Release Iteration is a macro-technique in that it uses a large set of learners in order to satisfy two requirements: 1) it gets the learning platform out as fast as possible, even though it may not be fully ready; and 2) it allows large scale testing of the platform before it is "polished."

A large and difficult or innovative project might use several Design Iterations and then make a Release Iteration. In turn, this process is repeated until the learning platform is completed.

After Action Review (AAR)

After running the iterations, use After Action Reviews, especially after you performa a Release Iteration, in order to transform deficiencies into actionable items. In addition, ensure you include the learners and their manager(s) in the AAR to ensure everyone is on the same track:

"Many years ago I was asked by a business unit leader to design a project management class with a significant emphasis on budgeting and forecasting. I complied with his request and designed several exercises intended to address this stated need. When the class ran, participants convinced the instructor that because they didn't have to do budgeting and forecasting, there was no need to spend much time on those subjects. Therefore the instructor skipped them. Participants (learners) were happy because they didn't have to learn content they didn't want to learn, and the instructor was happy because his end-of-class evaluations were extremely high. Unfortunately my client was angry. As he explained to me after the class ran, his employees were correct in saying that they didn't do budgeting and forecasting, which is why most of their projects were over budget and delivered late." — Larry Israelite in Lies About Learning.

AARs not only help you get to the root of problems by having the participants discuss the project in a non-threatening environment, but is also designed to help keep the learners, managers, and you on track so that everyone is striving for the same vision.

Agile Learning Design: Tools for Learners

2009-12-08T11:26:00.000-08:00

This is the sixth in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design
Post 3 - Orientation in Agile Learning Design
Post 4 - Designing Agile Learning
Post 5 - Selection in Agile Learning Design

In the last post I discussed some of the various media for delivering learning methodologies and objects. Before moving on to the final concept of "Iteration" in Agile Learning Design, I thought now would be a good time to discuss the tools the learners require for agile learning in a dL (distributed Learning) environment.

Networked Computer

While this may seam obvious to some, we need to realize that a lot of learners in the workplace do not have access to one or to one that is located in an environment conductive to learning.

Learning Management System (LMS)

And this does not mean access to any old LMS, but one in which the learners can easily search and locate the content they need. As Nicole Fougere pointed out in a recent blog post, Aaron Silvers took a show of hands at DevLearn and discovered this seems to be the last thing that we require of a LMS, but if the learners cannot easily use it and dislike using it, then it is absolutely worthless, no matter how many other bells and whistles it may have.

iTunes/iPod (or similar system)

While there is an adverse reaction to providing this set-up to learners in most organizations, podcasting is a proven and viable means of learning. Yes they may listen to music on their iPods, but they also use email, cell phones, and internet services for personal reasons. Lets provide them with the tools they need. To increase the capabilities of the device (mLearning), consider the iTouch.

Yammer or Twitter

For networking between various learning episodes. Note that one study found that 150 followers is the magic number.

A Guide for Managing Learning

Not everything is hi-tech. Perhaps the best guide around is Getting Your Money's Worth From Training & Development A Guide to Breakthrough Learning for Managers and Participants by Jefferson, Pollock, and Wick. This book is conveniently divided into two parts: 1) start at one end for managers or 2) flip the book over and it is for the learners. It is based on perhaps the best book on training: The Six Disciplines of Breakthrough Learning: How to Turn Training and Development Into Business Results.

PDF Reader

For reading and printing PDFs.

What would you include in your learners' toolbox?

Selection in Agile Learning Design

2009-11-24T07:55:00.000-08:00

This is the fifth in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design
Post 3 - Orientation in Agile Learning Design
Post 4 - Designing Agile Learning

The fourth concept of PODSI (Plan, Orientation, Design, Select, & Iterate) is selecting and developing the learning objects that best promotes learning. However, the selection of the learning objects in Agile Learning Design is not really about SCORM or AICC standards, but rather about providing the best learning environment that supports the Agile ethos — "Solutions that promote and speed the development of learning processes over comprehensive documentation."

This means chunking the contents into the most logical units that promote and speeds the learners' ability to learn and then packaging it into various media that provide the best delivery methods. Thus rather than breaking the material down into the smallest possible objects and tagging it for future reuse, it must be designed with the learners first in mind. A few examples:

Lectures

Lecturing in the classroom is over and if it is not over in your organization then you are wasting company resources. Now that does not mean "lectures" are finished because they can be quite an effective and efficient means to deliver information, ideas, stories, etc. And it does not mean putting them in a Captivate or Articulate elearning program that locks the learners to their desk. Rather it means putting them in an audio file, such as mp3, so that they can be uploaded to an iPod or similar device that allows them to be portable. The research is starting to show this as the best method for delivering them (Science Direct [pdf] or The Independent) and that learners prefer this method (New Scientist).

Reading Material

Rather than putting a lot of text into an elearning program that requires the learners to click the next button 10 to 30 times, put it in a PDF that is more suited for this type of content. This gives the learners several options for viewing it on screen or printing it if they desire, in addition to being able to add sticky notes. PDFs now allow the use of links, videos, and audio that greatly increase their versatility. Yes an elearning program allows you to ask questions as the learners proceed through the "click screens" but you can get just as good or better effects by asking the questions at the beginning of the PDF file (or where needed) so that you "preframe" the learner before they read the material. In one study the learners had a 10 percent better recall when they first tried to retrieve the answer before it was shown to them (see Scientific American - second page).

Question and Answers

A classroom environment allows the learners to ask questions; however, there are good tools that allow you to compensate for this in a dL (distributed Learning [only the L is capitalized]) environment — Twitter (free) or Yammer (if you need a more secure environment). These tools allow learners, instructors, experts, etc. to exchange short questions, answers, observations, etc. Does this mean we are trying to do away with the classroom? Nope...

Classrooms

Classrooms allow the learners to gain skills in task that can not be accomplished in a dL environment. So why not just perform all the learning in the classroom? The two main reasons are learning variability and cost. People learn at different rates, thus they need to be able to train at their own pace in order to master the material. For example, this chart shows the learning variability of a classroom session that was transformed to a dL environment (Sitzmann, Ely, 2009):

In theory, a classroom has to be long enough for the slowest learner to learn if you want them to master the material. In reality, classrooms are normally designed for the average learner, which means a large percentage of learners are wasting their time and another large percentage are not given the time to master the material. Thus the best environment for most learning is not in a lockstep (classroom) environment, but rather in a self-paced dL environment. Secondly, classrooms are normally one of the most expensive media for learning. ISD or ADDIE is often wrongly identified as a classroom creation tool, but as one who was formally trained in it, one of the first things hammered into us was to select a different delivery option if at all possible because of a classroom's lockstep nature and cost. The real reason for the classroom's dominant position are the people and/or organization behind the tool — they fail to stretch themselves to create a self-paced learning environment.

Blended Learning

This does not mean the classroom is dead as there are many things that must be taught in a classroom type setting or safe learning environment. In addition, classrooms actually enhance dL in that a blended learning environment composed of classrooms and dL increase learning by an average of 11% for both procedural and declarative knowledge (Sitzmann, Ely, 2009). There seems to be something almost magical about blending the interactive and social nature of classrooms with the self paced environment of dL. In addition, dL can have a attrition or drop-out rate as high as 20% as it often treats the person as a "lone learner." For example, in an informal learning episode a person will interact with an average of ten people (Tough, 1999). Adding blended learning and Twitter/Yammer provides the social aspect that we need when it comes to learning so that it does not become a lone-learning environment.

Doing

By delivering the pre-learning to the learners, a classroom type environment or similar setting can now be used exclusively for not only the required social interaction but perhaps more importantly for activities that allow the learners to perform (doing). In addition, as our online tools become more powerful, we are able to create activities that allow the "doing" in a dL environment. For example, although this multiplication game is made for schools, it has an interesting engine behind it:

html file : This of course is the file your browser opens.
Flash file (swf): Shockwave Flash file.
xml file: This is the file that the swf app reads. It supplies the "questions" and "answers." If you open it up with a text editor and study it, along with running the game, you will see that you can edit the xml file to create new questions and answers. While I have not tried it, you could probably add images to the questions/answers.
Note: You can download the zip file that contains the above three items.

It's the xml file that is the most interesting because rather than having to edit the flash file, a Learning Designer with a little training could edit the xml file to update it or rework it for a different learning environment. A few of these types of applications would make a good addition to an elearning development library that allow activities to be rapidly built. Has anyone seen other similar examples or instructions for creating them?

Building objects such as these are faster and easier than trying to incorporate everything into one giant elearning package. In addition, they can be iterated faster and are easier to fix and update. But most importantly, they consider the learners first by targeting their learning needs. Questions? → → http://twitter.com/iopt

References

Sitzmann, T. & Ely, K. (2009). Web-Based Instruction: Design and Technical Issues which Influence Training Effectiveness. Retrieved Nov, 2, 2009: http://webboard.adlnet.org/Technologies/Evaluation/Library/Additional%20Resources/Presentations/ASTD%202009%20Presentation%20Slides.pdf

Tough, A. (1999). Reflections on the study of adult learning. Paper presented at the 3rd New Approaches to Lifelong Learning (NALL) Conference, University of Toronto: Ontario Institute for Studies in Education, Toronto, Canada. Retrieved January 8, 2008 from http://www.oise.utoronto.ca/depts/sese/csew/nall/res/08reflections.pdf

Designing for Agile Learning

2009-11-19T07:52:00.000-08:00

This is the fourth in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design
Post 3 - Orientation in Agile Learning Design

The third concept of PODSI (Plan, Orientate, Design, Select, & Iterate) is Learning Design to facilitate interactions between humans and content in order to increase performance. It accomplishes "interactions" through the use of "awareness" that not only allows the content to sense and respond to the learners, such as feedback and guiding them to their next learning need; but also allowing the learners to sense and respond to the content; and as was noted during a Twitter conversation (with @usable learning and @Kathysierra), "the awareness should be more like Amazon's Lists rather than Clippy." Note that the definition is based somewhat on Safer's (2007) definition of interaction design.

Agile Design captures the texture & nuance of learning

Almost anyone can produce content but it takes a good Learning Designer to add awareness. It is also contextual in that it facilitates specific performance problems under a specific set of circumstances — my solution may not work for a similar problem. The end goal is to produce adaptive, agile thinkers, competent to perform within a dynamic working environment (Mark ley, 2006). While Learning Design is mostly art, it does has best practices.

Learning Design does not align itself with any one medium or technology, rather it is only concerned with the correct technology that aids in the learning/performance solution. Thus, it might be compared to distributed Learning (dL) that relies primarily on indirect communication between learners and instructors that allows the learners to learn at different times, at their own pace, as well as in different places. The old way of spelling the acronym was "DL", however this emphasized delivery method and learning equally, thus the correct acronym is now "dL", which emphasizes Learning without focus on delivery (Markley, 2006). That is, it uses face-to-face instruction when it makes sense.

Techniques to Learning Design

While there are specific methodologies for creating learning or instructional design, such as ISD, ADDIE, and van Merriënboer's 4C/ID Model; there are four design lenses or techniques that provide a means for viewing the overall structure of a specific learning design:

1. Performance-Centered Design

Focuses on the tasks that are composed of actions and decisions that the learners need to perform. A Learning Designer uses an Exemplary Performer as a model and then they build the instructional content and add awareness to it.

2. Guru Design

Focuses on the skills and knowledge of experts (SMEs), in which the designer may or may not be the guru. A Learning Designer uses one or more SMEs as knowledge sources and then they build the instructional content and add awareness to it.

3. Learner-Centered Design

Focuses on the needs and goals of the learners who guide the design; while the Learning Designer aids with the content and awareness. This is somewhat similar to user-centered design that is based on the concept that the people who use a product or service know what their needs, preferences, and goals are, thus they and the Learning Designer collaborate throughout every stage of the Agile Design process to build the content and awareness. It should be noted that the vast majority of so called "Learner-Centered Designs" out there are based on the other three design techniques because they are focused on what others thought the needs and goals of the learners should be, not what the learners thought they should be.

4. System Design

System Design focuses on the system's inputs, outputs, processes, feedback loops, goals, etc. to guide the design.

Specialty Designs (subset)

This includes ISD or ADDIE, which is basically a combination of Performance, Guru, and System Design, but normally little or no Learner-Centered Design (not because the model won't let you, but because the designers fail to). It also includes the micro-instructional designs, such a van Merriënboer's 4C/ID Model that focuses on task specific skills.

"The answer is, there's an infinite number of answers." - Amanda Palmer of the Dresden Dolls

Almost no Learning Design project is accomplished through just one of the four approaches or subsets, but is normally a mixture of them, with one of them being the primary approach to design. For example, a Learner-Centered Design might perform System Design and call on experts or gurus to help with the design; while a Performance Design might include some System Design, in addition to using Merriënboer's 4C/ID for some specialty tasks.

So just as you can "plug and play" different tools or methods into ISD, you also plug these tools into an Agile Learning Design so that rather than working with a tool box that only contains a hammer, you work with a full set of tools that compliments the learning platform in order to fast-track and retain learning.

Learning Design Approaches & Orientation

The source of where you get the content (Exemplary Performers, Expert Performers, SMEs, and/or Learners/Performers) as discussed in the last post clues you to the level of complexity of the design environment, which in turn tells you the primary design approach:

Exemplary Performers → Simple Environment → Performance-Centered Design
Expert Performers → Complicated Environment → Guru Design
SMEs & Learners/Performers → Complex Environment → Learner-Centered Design
Learners/Performers & managers → Chaotic Environment → System Design

Or which could be pictured as:

Click to enlarge

Being able to locate the correct level of complexity of the environment tells you the main design approach to take:

Simple Design Environment - SCR

Sense by using a collaborative process to create shared awareness and understanding of each team member's perspectives in order to create a mental model of the learning problem so that the correct decision-making can be performed.

You know you are in a simple learning design environment when you have Exemplary Performers who role model the required performance while you observe and categorize into tasks, skills, knowledge, and performance steps.

You respond by applying best practices such as creating learning objectives through a series of If/Then statements:

If we want to increase sales of our new service, then the sales representatives need to be able to perform an effective sales presentation. If we want them to perform the presentation, then they need to learn these skills __________, __________, and __________ (skills are categorized by observing the Exemplary Performers role modeling). If they need to perform these skills, then they will require this knowledge __________, __________, and __________ (knowledge is categorized by interviewing the Exemplary Performers role modeling).

This series of If/Then statements can also be visualized by using Performance or Action mapping as Catchy Moore shows in this slide presentation:

Design Lively Elearning with Action Mapping

View more documents from Catchy Moore.

Complicated Design Environment - SAR

A complicated learning design environment is similar to a simple learning design environment except rather than having Exemplary Performers who you observe, you have SMEs (Subject Matter Experts), who you interview and ask questions in order to analyze their responses.

You then respond by discovering patterns in their responses and transforming the information into good practices. And normally the only way to determine if it is indeed a "good practice" is through a series of iterations. Thus while a simple environment will only require a few iterations, a complicated environment will require several more.

Complex Design Environment - PSR

Since there are no Exemplary Performers to observe or SMEs to interview, the relationship between cause and effect can only be perceived in retrospect, thus the approach is to probe through deep collaboration among the learners, managers, and designers, such as telling stories about what they are experiencing (narratives). It is often helpful to look at the system and processes by starting with the output and working backwards through them in order see what to discuss (collaborate) and if it will help with the solution. Thus the primary design approach is Learner-Centered with the learners fulfilling the roles of SMEs, with perhaps some System Design added in. In addition, you can use a processes similar to the method Joe Deegan describes in his blog post, Project Based Learning in 3 Steps.

This probing effect should start to paint a picture or pattern that allows you to sense an "emergent practice" that can be responded to by designing and then implementing a solution based on the observed pattern. Since this will be a new practice, it will more than likely have to go though several rounds of iterations to arrive at the "emergent" practice.

Chaotic Design Environment - ASR

Since there is no relationship between cause and effect that the team (learners, managers, and designers) can agree upon, you will need to look at the system and processes by starting with the output and working backwards in order see what you can act upon. This might seem similar to a Complex Environment, but with a Chaotic Environment you are basically taking guesses of what to do (perhaps educated ones), while with a Complex Environment you are seeing patterns and getting an "Aha! moment" — this will work.

Once the change has been implemented, sense the environment again and see if the team can now agree upon the correct level of complexity. If not repeat the process with a new "act" until an agreement can be made.

By forcing changes into the chaotic environment you eventually push it into one of the other three domains. At this point a pattern should emerge that will allow the team to correctly identify the environment (more than likely a Complex Environment), thus you can now respond with one of the above three approaches.

The Complexity of Design Approaches

Knowing which design environment you are in helps with the planning by, 1) informing you of the number of design approaches that will be involved, and 2) estimating the number of iterations that will be needed.

1. As the level of complexity increases, the number of design approaches to solve the problem correspondly increases; however, there will normally be one major design approach. This will give you an idea of the scope of the design solution that you will be working in:

2. As the level of complexity increases, the number of iterations to reach a "good-enough" level correspondly increases. This will give you an idea of the number of iterations that will be needed:

Up Next

The design concept creates the basic plan for carrying out the Selection and development of learning objects for a dL platform, which will be covered in the next post. And while selection might seem rather mundane at first, it's more or less the heart of Agile Design.

References

Markley, J., 2006. The Army Distributed Learning Program. Training and Doctrine Command (TRADOC): presentation given at the U.S. Army Courseware Conference March, 14, 2006. Retrieved No, 2, 2009: http://wow.tradoc.army.mil/tadlp/presentations/dlcwconf06.pp3

Orientation in Agile Learning Design

2009-11-10T10:37:00.000-08:00

This is the third in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.
Post 2 - Planning in Agile Learning Design

The second concept of PODSI (Plan, Orientate, Design, Select, & Iterate) is Orientating to ensure you understand the contextual issues of the environment you will be designing in.

In the early 70's, three similar, but different, concepts emerged on the importance of orientation and learning — Boyd's OODA Loop, Double Loop Learning, and After Action Reviews.

Col. John Boyd, USAF, developed the OODA Loop for decision-making in a combat environment, particularly for combat pilots, and is now used in many organizations. He viewed problems as a cycle of Observation, Orientation, Decision, and Action, (OODA) and determined that whoever could cycle through the loop the fastest would win in a combat fight.

Boyd realized that people normally performed three of the decision-making tasks — observing, decision, and then performing an action, perhaps without really thinking about it, but one of their biggest weakness was failing to orientate themselves to the environment; thus he spent most of his time talking about it. This is why he highlighted "Orientation" in his diagram.

Chris Argyris coined the term "Double Loop Learning" and "Single Loop Learning. Single loop learning has often been compared to a thermostat in that it makes a "decision" to either turn on or off. Double loop learning is like a thermostat that asks "why" — Is this a good time to switch settings? Are there people in here? Are they in bed? Are they dressed for a colder setting? — thus it orientates itself to the present environment in order to make the wisest decision.

A person who is double loop learning is basically "orientating" herself to all possible solutions within her environment by asking a series of "whys" that is similar to Sakichi Toyoda's (the founder of Toyota) method who used a technique he called the Five Whys — when confronted with a problem you ask "why" five times. By the time the fifth why is answered, you should be at the root cause of the problem.

Amusing, but informative video on double and single loop learning (6.21 minutes):

Before the early 70's the U.S. Army used Performance Critiques to determine the effectiveness of training (mostly war games), which in a nutshell determined who won and who lost. Deciding that this was not the best way to get their money's worth they came up with the After Action Review (AAR):

The U.S. Army's did a slight twist on orientating oneself in that after a learning event, there is still more to learn by re-orientating oneself to the past and then asking a lot of whys so that new learnings are created — one prepares for the future by learning from the past. Learning is transformed from an event to a process by discovering "lessons learned."

Differences between a Performance Critique and an AAR:

Orientation in the Agile Design Environment

With all this emphasis on orientation you would think that by now we would have it down pat, but the truth is that people do not spend enough time on it, thus solutions continue to miss their intended mark. To aid us in orientating ourselves to the proper level of complexity so that the initial learning architecture can be designed, we will use the Cynefin (pronounced cunevin) framework.

The Cynefin framework was created by Dave Snowden and coworkers at IBM's Institute of Knowledge Management and consists of five domains:

Simple - the relationship between cause and effect is obvious to all; the approach is to Sense - Categorize - Respond by applying best practices.
Complicated - the relationship between cause and effect requires analysis or some other form of investigation and the application of expert knowledge; the approach is to Sense - Analyze - Respond by discovering patterns and then apply good practice.
Complex - the relationship between cause and effect can only be perceived in retrospect, but not in advance; the approach is to Probe - Sense - Respond by sensing emergent practice, such as telling stories (narratives), which is similar to an AAR.
Chaotic - there is no relationship between cause and effect at systems level; the approach is to Act - Sense - Respond to discover novel practice (take action) in order push the environment into one of the other domains so further action can take place.
Disorder - the state of not knowing what type of causality exists, thus people will revert to their own comfort zone when making decisions.

In this short video, Shawn Callahan of Anecdote gives a very good explanation of the Cynefin or Complexity model:

While gaining a full understanding of Cynefin framework requires a fairly extensive workshop, there are a few simple techniques that will help to identify the complexity of an Agile Design environment. However, to do so we need to tightly define three terms: subject matter expert, exemplary performer, and expert performer.

Subject Matter Expert (SME): Knows the subject or task, but does not presently perform in that area. An example is a college professor that teaches business, but is not engaged in a business; or a person that has performed in the subject area in the past in a wide variety of contexts, but is not presently a performer in that area.

Exemplary performer: Is able to perform the tasks for a certain subject area and is worthy of imitation, but does not have a great deal of knowledge about the peripherals surrounding the subject or task.

Expert performer: Is able to perform the tasks for a certain subject area and is worthy of imitation; in addition they have a great deal of knowledge about the peripherals surrounding the subject or task. An example might be a physician assistant (PA) who works in that job during the day and teaches college courses about it at night or a PA who not only performs the duties, but has performed in a number of surrounding areas that gives him a broad context of the subject and tasks. Basically an expert performer is both a SME and Exemplary Performer.

These definitions of the various "experts" that learning/instructional designers call upon are keys for identifying the complexity of the design environment:

Simple

In a simple design environment someone on the team has identified one or more Exemplary Performers who will be the role model(s) for the learning being designed. For example, a manager comes to you who wants to train her people to perform in the same manner as an Exemplary Performer she has identified; or during your research you identify a few Exemplary Performers who will make perfect role models.

While this is normally one of the easier learning platforms to design, it does have a couple of pitfalls. The first is thinking that since it is fairly easy to design, it is also easy to learn and perform, thus the failing to build enough practice time into the learning platform. Designers often become so absorbed in their work that they fail to realize how much time they are putting into it, thus they spend a couple of weeks working on the task, then think they can transform it into a two-hour information dump.

The second pitfall is failing to support the informal learning that must occur after the formal learning. There is an average of an 1:4 ratio in which one hour of formal learning produces four hours of informal learning. Thus support for the informal learning is also required to transform a training event into a learning process — the U.S. Army helped with this by giving us the concepts used in an AAR.

Formal learning is a seed that produces a lot of informal learning. If you don't plant the seed, you don't get the fruit. If you don't nourish the plant (informal learning), you end up getting underdeveloped fruit.

Note that the two pitfalls are not limited to a Simple environment, in that they can also occur within any of the other environments. However, in a Simple environment you will probably be more concerned with practicing within the formal learning environment than within an informal learning environment.

Complicated

In a Complicated design environment someone on the team will identify one or more Expert Performers whose skills (exemplary performance) and knowledge of the surrounding tasks and subject matter can be combined and/or transformed to form a "good enough" practice. The idea here is to take the "best" practices of their exemplary skills and subject matter expertise, along with the input of others, and combine them into a workable performance solution. In addition, you are going to start relying on the learners or affected (those who are going to be most affected by the learning solution) more as the problems are going to be slightly more wicked than within a Simple environment.

Like the simple environment, you will have to watch for the two pitfalls; however, in this Complicated learning environment you will probably have to pay about equal attention to both the formal and informal learnings of the learners/performers, in addition to relying on more iterations to ensure you get the feedback of the affected.

Complex

In a Complex learning design environment there are few or no exemplary performance examples to draw upon, thus you rely mainly upon subject matter experts (SMEs) and others to try to draw a picture of an "emergent" practice. However, some, if not most of the SMEs, should come from the environment (those affected) as they are probably the best "experts" of that environment.

While the initial learning platform might start out with more formal learning than informal learning, you need to look for solutions during the design and iterations to support informal learning while lessening the need for formal learning. This is because the complexity of the environment normally only requires a small seed of formal learning but needs extra nourishments of informal learning.

Chaotic

In a Chaotic learning design environment the initial solutions will come mostly through the managers and affected/learners. While the approach is to Act - Sense - Respond, it will more than likely require several iterations, rather than a one time shot. After each iteration, reexamine the environment to see if it has dropped to a lower level, more than likely a Complex environment, then take the appropriate approach.

We look at the present through a rear-view mirror. We march backwards into the future. - Marshall McLuhan

While you will often look to the past to predict the future, entering a chaotic environment is often uncharted territory, thus you will need an intense amount of collaboration to create unique solutions so that you do not harm your customers and the workers/learners with a poorly implemented solution.

Disorder

If by chance you discover you are in a Disorder environment where no one is sure of the environment, then more collaboration is called for with perhaps a few more "experts" (mostly the learners). Remaining in this state is unacceptable.

Agile Matrix for Orientation

I took the main points (practices) of this post and inserted them into the Value and Principles Matrix (See first post on Agile Design for Learning) to show how orientation fits into the values and principles of Agile Design:

Download Excel Matrix file for Orientation (xls version): Agile_Matrix4.xls
Download Excel Matrix file for Orientation (xlsx version): Agile_Matrix4.xlsx

Note On some systems the xlsx version will try to download as a zip file. In that case, click the above xlsx file with the right mouse button to bring up the context menu and then click "Save Target As..." item. When the dialog window opens, change the extension from .zip to .xlsx — this will save the file correctly.

Planning in Agile Learning Design

2009-11-03T08:12:00.001-08:00

This is the second in a series of posts on Agile Learning Design:

Post 1 - Agile Design: An Ethos for Creating Learning Platforms.

The first concept of PODSI (Plan, Orientate, Design, Select, & Iterate) is planning to ensure the goal or target is identified and that all stakeholders see the feasibility of the project. One of the most common mistakes with designing learning processes is failing to link the learning platform with a business need — the business unit or customer does not understand how the performance solution links to their business needs and/or the designers fail to link the correct solution to a real business need.

Business linkage is a "high value add" that is defined as the difference-making in business because it adds high value (Garnevale, Gainer, & Villet, 1990). Yet, defining how our learning processes and platforms link to other business units is one of the activities that we normally spend the least amount of time on (Trolley, 2006). One way to understand your customers' need is to determine how they will evaluate the effectiveness of the learning solution before you begin the project, not after.

Another impediment to identifying the correct business linkage is the failure to bring the learners into the planning stage. Rittel (1972) noted that often the best experts with the best knowledge for solving wicked problems are those affected by the solution; in this case it is the learners themselves. Yet, the only time we normally bring them in is to be guinea pigs for testing our learning process. Now normally you will not be able do bring the entire population of them in, but do bring in enough learners that will actual represent the population.

The learners are the real stakeholders, thus even if you or the managers don't agree as to what they are saying, you need to listen, guide, and act on their needs and perspectives so that they take ownership of the learning and performance solution. This is one way they gain metalearning and metacognitive skills.

In addition, Rummler and Brache's experience was that 80% of performance problems reside in the environment, such as processes and systems, so ensure the problem is really a learning/training problem, not some other performance problem; while most of these problems do require some sort of learning solution, ensure you get to the root cause.

Note that Rummler and Brache's 80% rule could differ greatly from yours; if the performers/learners have a great deal of autonomy, then you would expect a larger percentage of performance problems will be in the skills and knowledge area (learning solutions), rather than the environment (other performance solutions) as there is less of a chance that an autonomous performer is restricted by a process or system. Another reason is if the performers/learners work in a highly evolving or complex environment that require unique solutions from them, rather than relying on a process that leads them to a preplanned solution.

Since designers, managers, learners, and perhaps some subject matter experts and/or exemplary performers will be in on the planning, a high degree of collaboration needs to take place to accurately identify the problem and solution. Collaboration does not mean agreeing with everything others say as this leads to group-think or the Abilene Paradox. You want the team members to disagree and share information. To encourage lower status members to share, which may often be the learners, the members' expertise needs to be acknowledged to the group at the onset of the planning stage — people who sense they have a high status in the group will more likely want to share their knowledge.

Note that by arguing, we mean focusing on the problems, not crass behavior; telling narratives, not finger-pointing; and finding solutions that benefit the organization, not trying to force one's agenda. No matter how complex or argumentative an issue is, it can normally be broken down to three basic artifacts:

Questions
Ideas
Arguments (pros and cons)

Paul Culmsee tells a great story about using these artifacts to guide the discussion and collaboration in a blog post.

One of the tools for capturing the ideas during planning is a concept or mind map. While Paul's story uses Compendium (free), just about any mind mapping tool will work; however, Compendium was built especially for this type of collaboration in order to show different viewpoints, positions, and pros and cons organized by using logical connections:

For other tools that will aid the collaboration process, see McKinsey & Company's Using technology to improve workforce collaboration. In addition, the article includes a neat Flash based collaboration types and tools app that shows day-to-day activities with various technologies that promote work and learning flows.

Since this is an "agile" environment, we have to ensure that all the individuals are able to interact in a manner so that everyone understands the goal or target that needs to be met — exactly what change of performance will occur after the learners return to their jobs and how that change benefits not only the business unit, but the entire organization as well.

While I've given a few practices to support planning in Agile Design in this post, they can be supplemented by other practices, such as the Analysis phase in ISD, as long as they support the values and principles of Agile Design. This can best be checked by determining if they fit in the Value and Principles Matrix:

For example, I took the main points (practices) of this post and inserted them into the Value and Principles Matrix:

Download Excel Value and Principles Matrix file: Agile_Planning_Matrix.xls
Download Excel Value and Principles Matrix file: Agile_Planning_Matrix.xlsx

A final point is that planning is not a one-shot affair, but is iterative, thus it can be returned to on a as needed basis. For example, the next concept, Orientate (which will be covered in the next post), will often have to be performed before the initial planning can be initiated and/or once the learning solution is ran thru its iterations. Thus planning is not rigid, but follows the Agile values' of evolutionary and adaptive to ensure customer and learners' needs are met.

References

Garnevale, A., Gainer, L., & Villet, J., (1990), Training in America: The Organization and Strategic Role of Training. San Francisco: Jossey-Bass.

Rittel, H. (1972). On the planning crisis: systems analysis of the "first and second generation.Bedriftsokonomen. No. 8, pp.390-396.

Trolley, E. (2006). Lies About Learning. Larry Israelite, ed. Baltimore, Maryland: ASTD.

Agile Design: An Ethos for Creating Learning Platforms

2009-10-26T08:20:00.000-07:00

Software design and related practices and methods have had a significant influence over the Instructional Design field. For example, ADDIE, Dick and Carey, and Rapid Prototyping are heavily influenced by software development methodologies (Rawsthorne, 2005). Software design methodology is now going through another paradigm shift — Agile Design. And rather than being a methodology, it is more a philosophy or ethos that is best described by its manifesto (Agile Alliance, 2001):

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

The Agile approach recognizes the need for collaboration, faster design solutions, feedback and change for producing business value in our ever faster and more networked society. Thus, for learning professionals to keep pace with the rest of the organization, Agile Design could easily be adapted to fit the needs of the learning and training community by providing an ethos for the design of learning:

We are uncovering better ways of designing
learning processes by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Solutions that promote and speed the development of learning processes over comprehensive documentation
Customer collaboration over contract and formal negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Because we still value the items on the right means that we do not have to abandon the technologies that make up our profession, such as ADDIE, 4C/ID, ARCS, Captivate, and PowerPoint. But rather we pull the best concepts from them that will support the values and principles of Agile Design.

Values and Principles of Agile Design

Since Agile is a more of a philosophy, it has values and principles that guide its practices. The Sidky Agile Measurement Index (SAMI), developed by Sidky & Arthur (2008), is probably the most widely used method for guiding Agile principles. It is composed of five values: communication, collaboration, evolutionary, integrated, and encompassing. These five values were heavily inspired by three of Malcolm Gladwell's ideas in The Tipping Point:

People = communication & collaboration
Message = evolutionary, integrated, & adaptive
Suitable environment = all encompassing

Listed below are the five values with their descriptions (please note that I changed the descriptions to fit learning rather than software development):

Encompassing: Establishing a vibrant and all-encompassing environment to sustain agility
Adaptive: Respond through change through multiple levels of feedback
Integrated: Develop high quality learning solutions in and efficient and integrated manner
Evolutionary: Deliver learning processes early and continuously
Collaborative: Enhance communication and collaboration

The Agile Manifesto basically outlines 12 principles; however, Sidky & Arthur (2008) discovered they could group them into five tight principles (please note that I changed the descriptions to fit learning rather than software development):

Embrace change to deliver customer value
Plan and deliver learning processes frequently
Human centric
Technical excellence
Collaboration with business people

These five values and principles can be placed in a matrix to guide the selection and population of practices that will best achieve the ethos of Agile Design. The matrix shown below lists the five values in the left column and the five principles in the top row. I then listed some Learning Design practices, concepts, and processes that may be used to guide a performance project. Note that the principles may vary from organization to organization and may even change from project to project within an organization, but any adopted practices should always be guided by the values and principles; that is, they should never go against them:

So that you don't have to reproduce the above matrix, I am including the Excel file for the Agile matrix of values, principles, and values. The xlsx file is for the latest version of Excel and is the one shown above. The xls file is for older versions of Excel and is the same except the colors are brighter and may need to be toned down:

These value and principles make Agile more adaptive rather than predictive; and people-oriented rather than process-oriented (Fowler, 2003). It is misleading to view it on the opposite end of a spectrum from "plan-driven" or "disciplined" methods as it implies that agile methods are "unplanned" or "undisciplined." A more accurate distinction is that methods exist on a continuum from "adaptive" to "predictive" and agile methods lie on the "adaptive" side of this continuum (Boehm & Turner, 2004):

To achieve an adaptive and people-oriented process, a strategy is implemented that allows collaboration among the designers, business unit (customer), learners, exemplary performers and/or SMEs, and other interested parties. To accomplish this, a conceptional framework is initiated that allows the strategy to carried out — Plan, Orientate, Design, Select, & Iterate.

Plan, Orientate, Design, Select, & Iterate (PODSI)

Plan by identifying the potential target, vision, and feasibility of the project that will ensure the active participation of all stakeholders. Determine if the managers are indeed going to collaborate or are willing to learn to collaborate. If they simply want you to be an order-taker then, "run my friend run, run as fast as you can!" Find a project with people who desire to collaborate.
Orientate in order to recognize the level of the complexity of the environment (Cynefin) so that the initial learning architecture can be started to solve the problem. Use Exemplary Performers and/or Subject Matter Experts to help identify the complexity of the environment.
Design by using a collaborative approach or model so that only the minimum required knowledge and skills are taught that will resolve the problem. Build other useful benefits into the learning process during the final iterations.
Select the correct learning objects, processes, and tools that will provide the needed knowledge and skills that support both formal and informal learning — the use of small learning objects will increase the speed of iterations and allow you to more easily transform parts of the instruction into informal and nonformal learning.
Iterate by prototyping the initial design and to determine what other performance support technologies are required that will fully support the learners' quest to better performance. Use After Action Reviews to transform deficiencies into actionable items. Transform the formal learning objects to informal or nonformal learning as possible.

"Failure at an organizational level seems to come from the inability to customize processes and make them their own. Trying to apply someone else's template to your organization directly doesn't work well. It leaves out too many important details of the previous successes and ignores your company's specific situation." — Kent Beck (2006 interview with InfoQ)

PODSI is dynamic in that the above stages are not step-by-step, all encompassing solutions but rather selected concepts from our discipline that best support Agile. Even though they may be performed in order, particularly for the first iteration, the concepts should be thought of more as a network, rather than a flowchart or template. Thus, while the last concept is to iterate the learning process in order to achieve the best solution, the other concepts are also iterated throughout the life-cycle of the project on an as-needed basis.

Essence of Agile Design

Thoughts?

Throughout the upcoming weeks I hope to expand on PODSI, thus I am interested in hearing your feedback, thoughts, and ideas. Please feel free to share, rip, and mix.

Note (December 23, 2009) The compleste series has been posted:

Agile Design: An Ethos for Creating Learning Platforms

Planning in Agile Learning Design

Orientation in Agile Learning Design

Designing Agile Learning

Selection in Agile Learning Design

Agile Learning Design: Tools for Learners

Iterations in Agile Learning Design

Periodic Table of Agile Learning

References

Agile Alliance (2001). Manifesto for Agile Software Development. Retrieved on June 28, 2009 from http://www.agilemanifesto.org/

Boehm, B.; R. Turner (2004). Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA: Addison-Wesley. pp.165-194

Fowler, Martin (2003). The New Methodology. Retrieved on June 28, 2009 from http://www.martinfowler.com/articles/newMethodology.html

Rawsthorne, P. (2005). Agile Methods of Software Engineering should Continue to have an Influence over Instructional Design Methodologies. Cape Breton University & Memorial University of Newfoundland. Retrieved on June 28, 2009 from http://www.rawsthorne.org/bit/docs/RawsthorneAIDFinal.pdf

Sidky, A. & Arthur, J. (2008) Value Driven Agile Adoption: Improving an Organization's Software Development Approach. Fujita, H. & Zualkernan, I. (eds). New Trends in Software Methodologies, Tools and Techniques: Proceedings of the seventh SoMeT_08. Volume 182. Oct 15, 2008. P149-164. The Netherlands: IOS Press. Retrieved Oct 22, 2009: Google Books

The Brick & Mortar Mix of Learning

2009-10-21T14:25:00.000-07:00

In his blog, Jay Cross points to relative old, but interesting article on informal learning: Informal Learning and the Transfer of Learning: How Managers Develop Proficiency and notes, "If you're still relying on formal training to develop managers, you might want to give this one a read."

The paper is more interesting for what it omits, rather than what it purpose seems to be. That is, it seems to tout the importance of informal learning, rather than from what I see is the real key finding — its the mix that matters. For example, it makes several comments along these lines:

"Because skills learned informally are likely to share similar features with transfer tasks in terms of context and content, the potential exists for skills learned informally to be more readily transferred than skills learned in formal training contexts."

"Our study suggested that managers learn mostly from informal learning, that proficiency is the product of informal learning, and that metacognitive knowledge and self-regulation skills moderate informal learning and the transfer process.

In the paper they show the following chart (p377):

When referring to the chart they note, "The distribution indicates that managers reported learning all twenty skills predominantly from informal learning activities." Yes, while the managers believed they learnt more from informal learning, the chart actually seems to be showing that they learn a core base from formal instruction, and then they build from their proficiency from there. In addition, some core skills only need a drop of formal learning to get the process going, while others require a heavy dose.

This goes back to the previous post in which I noted that some learning episodes that are strictly informal may be too narrowly based in that the learner only learns part of a task or superficial skills that may not be transferable to the job (Bell and Dale 1999).

Thus, just as we have a "blend" of learning media and processes, we also need the proper mix of formal and informal learning. This means you not only have to select the proper blend of formal learning, but also select the proper mix of formal and informal learning. In turn, you then have to select the best blend of informal learning that will help the learners transfer their skills to the job.

In the paper they mention a study in which pilots with more flight experience perform better on a simulated flight test (that is, a transfer task) than did their novice counterparts. Now I don't believe that anyone is going to argue this point, but the other part to it is that those better performing pilots would have never been able to perform in the first place if it was not for their core skills gained with formal learning.

In the comment section on my previous post Michael Hanley notes, "...the reality is that all of these exist on a 'learning continuum.'" This learning continuum is also the subject of a post by Clark Quinn: The Formal/Informal Continuum.

Thus, its not a matter of designing learning from one side of the continuum or the other because you need the core skills from one side and the proficiency of actually being able to put those skills into practice from the other side. In addition you need that mix from the middle that is not readily identifiable as either formal or informal.

Informal Learning, huh, yeah, what is it good for?

2009-10-20T08:23:00.000-07:00

Robert Bacal's post, "What Do Intellectually Impoverished Educators/Trainers Do To Make A Living? Why They Make Up New Fancy Sounding Terms" attempts to be a call to action to go beyond the hype, but turns quickly into what seems to be almost a hyperbole of fear. While Robert writes of informal learning, learning 2.0, education, and training; I'm going to stick with informal learning in the workplace to help keep this post more focused.

First, there seems to be some confusion as to the origin of the term "informal learning" by both the poster and commentators. While Jay Cross brought the concept to its present level of popularity, Malcolm Knowles is generally considered to be the originator of the term through his book published in 1970: Informal Adult Education: A Guide for Administrators, Leaders, and teachers.

Robert makes a feeble attempt to define informal learning, but basically makes no sense at all — "informal learning simply refers to learning that occurs....well, informally."

Actually both informal and formal learning has nothing to do with the formality of the learning but rather the direction of who controls the learning objectives or goals. In a formal learning environment the training or learning department sets the goals and objectives, while informal learning means the learner sets the goals and objective (Cofer, 2000).

In addition, if the organization (other than the training department) sets the learning goals and objectives, such as a line manager directing OJT, then it is now normally referred to as "nonformal learning" (Hanley, 2008). Thus in a formal learning process, learning specialists or trainers set the goals, while a nonformal one has someone outside of the learning department setting the goals or objectives.

Robert mentions the terms "incidental learning" and "intentional learning," which basically refers to the intent of the learning objectives. An intentional learning environment has a self-directed purpose in that it has goals and objectives on what and/or how to learn; while incidental learning occurs when the learner picks up something else in the learning environment, such as the action of a model, that causes him or her to loose focus on the learning objectives or goal and focus on an unplanned learning objective.

While incidental learning is often dismissed by trainers, it is an important concept because it often has a motivating effect with the learners that leads to "discovery" learning. So unless other considerations prevent it, it can be worthwhile to detour from the primary objectives to take advantage of an unplanned "teachable/trainable moment." For example, if I'm instructing the learners to operate forklifts and we are discussing safety concepts, one or more of the learners might become interested in a safety concept that is unrelated to the operation of forklifts. However, if possible I should try to help them with the unrelated concept, which in turn should help to motivate them with the related safety concepts pertaining to forklifts. In addition it could lead one or more of them to become more interested in the safety program and perhaps lead them to become more involved with it.

If we were to map out the above types of learning it might look something like this:

While it might seem obvious to most readers that both formal and informal learning include both incidental and intentional learning, it might not be as obvious that formal learning often includes episodes of informal learning and vice versa. A two-year study calculated that each hour of formal learning spills over to four-hours of informal learning or a 4:1 ratio (Cofer, 2000). Thus Bell used the metaphor of brick and mortar to describe the relationship of formal and informal learning. Formal learning acts as bricks fused into the emerging bridge of personal growth. Informal learning acts as the mortar, facilitating the acceptance and development of the formal learning. He noted that informal learning should NOT replace formal learning activities as it is this synergy that produces effective growth.

And of course the opposite also occurs in that episodes of informal learning often leads to formal learning. In addition, some learning episodes that are strictly informal may be too narrowly based in that the learner only learns part of a task or superficial skills that may not be transferable to the job (Bell and Dale 1999).

The Hype of Informal Learning

Robert does make a good point when he writes, "people going around trying to convince corporations that classroom learning is wasteful, and that they should be pouring money into informal learning activities."

As stated earlier, "informal learning should NOT replace formal learning activities as it is this synergy that produces effective growth." Yet it seems that some informal learning proponents still wish to do away with most formal learning processes. For example, they will list several references that claim that only 10% to 20% of formal learning processes actually transfer to the job (Cross, 2007, p. 32), but they fail to check any of those references — if they did, they would find it is based on a theoretical question rather than any real research.

Another fallacy is claiming OJT as part of informal learning in order to increase its importance and raise its percentage of the total learning. In order to claim OJT and other organizational directed learning, they would have to redefine informal learning in basically the same manner as Robert does — informal learning refers to learning that occurs informally.

Perhaps the newest hype or bandwagon is claiming that 80% of learning is informal and 20% is formal but paradoxically training departments spend 80% of their budget on formal learning and 20% on informal learning. Yet they fail to mention that a large portion of formal learning is informal learning that has been transferred to the learning department because it is difficult or inefficient to learn in an informal learning environment or because it is an important part of a process that cannot be left to chance. Thus its almost a dammed if you don't spend money on informal learning and damn if you do because now it is no longer informal but rather formal learning — because now you, well, uh, "formalized" it!

In addition, a lot of informal learning is created through formal learning process — remember the 4:1 spill over ratio? Now while we do need to support informal learning processes more, lets not attack the very thing that helps to create informal learning and synergizes well with it. In fact this is probably one of the main reasons that training fails to transfer to the job — trainers and learning specialist fail to follow through after the training event. Training/learning is a process; just as Tom Peters urges leaders to manage-by-walking-around, we also need to walk-around and help with the informal learning that is required for the formal learning to fully transfer.

Why Should We Invest in Informal Learning?

Just because informal learning has been hyped does not mean it has little or no value. As already discussed, there is a close synergy between formal and informal learning in that neither is very effective for many types of learning processes.

The real power of informal learning seems to be based with its close ties with social learning. Allen Tough, one of the pioneers of informal learning, discovered that learners interact with an average of ten or eleven people during an informal learning event (1999). He also noted that there may actually be more social interaction in an informal learning event than there is with a similar one of formal learning.

It is these multiple and repeated social and self learning episodes in the process that makes informal learning a powerful tool. That is, the number of connections greatly increases the chance for an idea, value, or pattern of behavior to be passed from one person to another (Pontus, Magnus, & Kimmo, 2009). Thus information can be both abandoned readily and reacquired if later proved useful.

In a quite interesting story, Marcia Conner reports how Humana uses simple social media tools to engage with people across the organization in order to learn from each other. These tools further increase the number of connections — if individuals can learn many times, successful learning occurs regardless of the transmission pattern (Pontus, et al., 2009). This concept is similar to overlearning — practicing well beyond the point of initial mastery.

Since this is starting to lead to the next hype on Robert's list — learning 2.0 (self-directed learning using social networking and collaboration tools or informal learning on steroids) — it's time to sign off.

References

Bell, J., and Dale, M. (1999) Informal Learning in the Workplace. Department for Education and Employment Research Report No. 134. London, England: Department for Education and Employment, August 1999.

Cofer, D. (2000). Informal Workplace Learning. Practice Application Brief. NO 10. U.S. Department of Education: Clearinghouse on Adult, Career, and Vocational Education.

Cross, J. (2007). Informal Learning: Rediscovering the Natural Pathways That Inspire Innovation and Performance. San Francisco: John Wiley & Sons, Inc.

Hanley M. (2008). Introduction to Non-formal Learning. E-Learning Curve Blog. Retrieved October 19, 2009: http://michaelhanley.ie/elearningcurve/introduction-to-non-formal-learning-2/2008/01/28/

Pontus, S., Magnus, E., and Kimmo, E. (2009). Repeated learning makes cultural evolution unique. PNAS, 2009, 106 (33), p. 13870.

My iPod Learning Mix

2009-10-15T10:48:00.000-07:00

Since Jeff shared his great 10 Killer Content Sources for Your iPod Learning Mix, I decided to share a few of mine.

Note that the links will take you to their web site with RSS feeds or straight to their RSS feeds. If you wish to use iTunes to manage your feeds (as I do), open iTunes, click on "iTunes Store", click on "Podcasts" (Top of window), enter the title of the podcast into the search field and when iTunes list it, click the subscribe button.

This Week in Tech - MP3 Edition

This is perhaps my favorite podcast as Leo Laporte does a marvelous job of leading a panel of experts, such as John Dvorak or Kevin Rose (Digg fame) to discuss the latest on technology. This is one of the more interesting ways to learn as you feel you are at the kitchen table with a bunch of experts.

GROKS Science Radio Show and Podcast

Dr. Charles Lee and Dr. Frank Ling interview an expert a week on a science topic. These guys are smart interviewers as they always study up on the subject before the interview.

ASTD T+D podcasts

ASTD's podcasts are normally based on their magazine articles, thus are normally read to you. While this format does lead to a less engaging listening experience, it does help you keep up with the world of training/learning in that you can listen while walking or riding -- my preferred method for listening to podcasts.

Brain Science Podcast

Dr. Ginger Campbell takes you through the wonderful world of the brain, either by interviewing others or through her own experiences.

On the Media

The National Public Radio helps you make sense of the latest news by doing in-depth studies and criticisms of the sources behind the latest stories.

TedTalks (audio)

While most of you are probably familiar with their videos, you can also keep up with Ted through their podcasts.

Xyleme Voices

While they normally only podcast about every three or four weeks, they do come up with a few gems, such as an interview with Clive Shepherd the episode before last.

NPR: Story of the Day Podcast

NPR is one of my favorite news sources and this podcast spotlights some of their best stories.

BBC Digital Planet

More on technology from a British perspective.

This American Life

Chicago Public Radio presents their award winning show with master story teller Ira Glass.

Data & Training, Learning, Folksonomy, Scenario Learning, Outsourcing, & Rapid eLearning

2009-10-12T08:13:00.000-07:00

Training to Climb an Everest of Digital Data - New York Times

Science these days has basically turned into a data-management problem.

For the most part, university students have used rather modest computing systems to support their studies. They are learning to collect and manipulate information on personal computers or what are known as clusters, where computer servers are cabled together to form a larger computer. But even these machines fail to churn through enough data to really challenge and train a young mind meant to ponder the mega-scale problems of tomorrow.

Learning new tricks improves wiring in the brain - Reuters

"We tend to think of the brain as being static, or even beginning to degenerate, once we reach adulthood," said Heidi Johansen-Berg of Oxford University's department of clinical neurology, whose study was published in the journal Nature Neuroscience on Sunday. "In fact we find the structure of the brain is ripe for change. We've shown that it is possible for the brain to condition its own wiring system to operate more efficiently."

Folksonomy folktales - KM World

The Dewey Decimal System is not a good example of a taxonomy. Folksonomies are the exact opposite of the wisdom of crowds. Hierarchies are not rigid, conservative and centralized.

Scenario Based Learning - Speak Out

A SlideShare presentation (20 slides)

The U.S. Is Outsourcing Away Its Competitive Edge - Harvard Business Blog

You focus on R&D and turn over the low-margin commodity manufacturing to contractors. You make out like a bandit because you have the intellectual property and your contractors have so much competition they cannot afford to charge you more. All this assumes your manufacturing partner is content to subsist on your table scraps. But what if they have their eye on the prime rib, too?

There's nothing rapid about Rapid eLearning - VMG

However, in today's times, it's worth understanding that rapid just isn't what it says it is.

As a testament to this, I did some quick research and found four studies over the past seven years to demonstrate the reality of the situation. Interestingly, the findings are very similar (see below for details): the time to create one full hour of an intermediate level, Captivate/Articulate style elearning product is around 200-250 hrs.