Mapping the Performance

Part of the analysis phase is capturing the skills required for performance. While there are several methods for capturing these performance, ISD normally only lists Behavioral Task Analysis. However, many task are largely overt and nonprocedural in nature, thus they require a Cognitive Task Analysis. The solution of course is to simply plug the desired method or tool into the ISD or ADDIE model as it is quite dynamic rather than the stale linear model that some believe.

Four analysis tools will be discussed in this post that may be plugged into ISD:

• Behavioral Task Analysis
• Information Processing Analysis
• GOMS Analysis
• Critical Decision Method

Considerations for Analysis Tool Selection

Selecting the correct analysis tool is dependent on the type of actions the worker must perform. This performance is normally composed of two types of actions:

• Overt - behavioral and observable
• Covert - mental and not observable

While some some tasks are only composed of one or the other, more complex tasks may be composed of both actions.

In addition, the selection of the analysis tool is also dependent upon the task steps:

• Procedural - the steps are performed in order and are normally largely overt actions
• Rule Based - the steps do not have to be performed in a temporal order and are normally largely covert (cognitive) actions

Procedural Analysis Methods

These methods are used when there is a temporal order of involved steps, thus there is a set procedure for performing the task. Two analysis tools that fall under Procedural Analysis are Behavioral Task Analysis and Information Processing Analysis.

Behavioral Task Analysis

Behavioral Task Analysis is used to capture overt actions by observing and recording an Exemplary Practitioner perform the task. Questions may also asked to ensure the analyst has fully captured the performance. This is perhaps the easiest method. The output is a list of steps that may also have diagrams or pictures of the desired performance or behavior. A short example is:

1. Turn on computer and start spreadsheet application.
2. Load projected sales report spreadsheet template (prosale.exl).
3. Enter projected sales figures into designated spreadsheet cells.
4. Run spreadsheet macros.
5. Save file under new name, “pro***.exl”, with *** being the next sequential number For example, pro135.exl. Note: Do NOT overwrite template.
6. Forward to Planning Manager by email.
7. Exit application.

Depending upon the learners' prior knowledge and the complexity of the task, the list might also contain substeps as shown in this example:

1. Turn on computer
2. Start spreadsheet application
• 2.1 Click the Start icon
• 2.2 Scroll through the menu list and select the Excel application
3. Load projected sales report spreadsheet template (prosale.exl).
   • 3.1 Click on the file menu
   • 3.2 Click on the Open option
   • Etc., Etc.

Information Processing Analysis

This tool is used when there are both overt steps that require a set order and covert steps that require decision making of a yes or no nature. The overt actions are captured by observing and recording an expert performer, while the covert actions are captured by having the expert performer talk about their actions (thinking aloud). Capturing the decision making process makes it more difficult than the Behavioral Task Analysis, but it is perhaps the most common method as the majority of tasks require decisions. The output is commonly a flow chart composed of three elements that outline the task steps:

• Boxes - overt or covert actions
• Diamonds - decisions
• Arrows - order of steps

A short example of a flowchart showing a decision a forklift operator must make when moving goods from the receiving dock to a storage area may look like this:

Rule Based Analysis Methods

These methods are used when there is NO temporal order of involved steps, thus there is not a set procedure for performing the task. In addition, most of the task steps are normally of a overt nature. Two tools that fall under Rule Based Analysis are GOMS Analysis and the Critical Decision Method.

GOMS Analysis

The GOMS tool analyzes a task by examining four elements of it:

• Goals represents the intention to perform a task its underlying structures, such as subtasks, cognitive operations, and physical operations. For example, an Instructional Designer may have a task of selecting activities that will give a new sales person the skills to sell a product. That task includes subtasks of analyzing the skills needed, analyzing activities, etc. The ID will use cognitive operations, such as selecting an activity that actually teaches the skill and determining if the chosen media can effectively deliver the activity.
• Operations represent the physical actions, such as binding a learner's manual or pressing a button. It also includes mental operations, such as retrieving from memory or setting a goal.
• Methods represent sequences of operations that accomplish goals or objectives. It includes high-level goals that breaks a tasks into subtasks and low-level methods that are the actions that actually perform the subtasks.
• Selections Rules represent the context for selecting a particular method. That is, there may be several ways to accomplish a goal or task, but one may be chosen because it should perform this particular task the best (heuristics).

Capturing the performance of a covert task can be done in many ways and normally several are used to capture the task. Some of the more common ones are interviews, job shadowing, having an expert performer think-aloud, and storytelling.

The GOMS analysis can normally best be represented by a concept or mind map. The Goals will normally be placed on the map first, which is then followed by placing the Methods and their Selection Rules on the map. The details (Operations) are then listed. Shown below is a partial mind map of selecting an analysis tool:

Depending upon the size and the scope of the task, you might have to link or reference other documents that go into more specifics about the Operations, Methods, and Selections rules. However, be careful as one of the common mistakes in GOMS analysis is getting too specific, which results in long and detailed procedural descriptions that are difficult to follow.

Critical Decision Method (CDM)

This can be thought of as a Case Study, however, it also includes a visual reference or map. Just as a case study uses an actual incident to tell a story, CDM is also performed by having an expert tell a story about a particular task they performed in the past. For example, an Instructional Designer might tell how he developed a Just-in-Time program for training sales persons to sell a new product or a fire fighter might tell about the actions she took in fighting a gas station fire. Note that when the person tells the story, the interviewer might have to probe to gather some details. While the output might include a case study, it should normally include a map similar to this:

The CDM process goes like this:

• Sweep 1 - Identify a incident. It should come from a decision maker who was involved rather than a witness.
• Sweep 2 - The expert tells his or her story. Identify the key decision points and when they were made.
• Sweep 3 - These is where you “deepen” the interview by asking for analogs, mental models, other options, experience or training that was helpful, etc.
• Sweep 4 - Finally ask “What If” questions, such as “If the situation had been different what would have happened?”, or “What if a novice had been in charge?”

Key Points

The finale outputs, such as a list of task steps, flowchart, mind map, or CDM chart prove not only invaluable to the the design team, but may also make excellent performance or learning aids. Just as it is as important to use a correct analysis tool, it is also just as import to represent the findings in a manner that others can understand.

The final output might not be just one chart, but rather a combination of them as shown in the picture at the top of this post. Also the chart does not have to be combined into one page if it gets too cluttered, but may rather be composed of a collection of documents that are linked together electronically or referenced if they are printed. In addition, other types of charts and visualizations representations may also be used.

More detailed information can be found in two books, van Merriënboer's, Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training (1997) and Crandall, Klein, & Hoffman's, Working Minds: A Practitioner's Guide to Cognitive Task Analysis (2006). Merriënboer discusses Behavioral Task Analysis, Information Processing Analysis, and GOMS Analysis (plus other topics related to complex tasks), while Crandall, Klein, & Hoffman discusses Concept Mapping and Critical Decision Method (plus other topics related to Cognitive Task Analysis).

Both Merriënboer and Crandall, Klein, & Hoffman's books are excellent references and provide different information about Cognitive Task Analysis, however, Merriënboer's is now out of print, thus the used books may be a bit on the pricey side.

ADDIE Does More Than Classrooms

One of the misconceptions of ISD is that it was created to only build classroom training environments.Yet, one of the old Army manuals (1984) that is used for training ISD shows the above options for training. It also notes that options 1, 2, or 3 should be used in lieu of classroom training if it can adequately perform the job. Thus, classroom training should normally be the last option if there is more than one viable option. This is because classroom training is normally one of the more expensive options.

The five options are shown below with a few notes about them:

1. Job Performance Aid (JPA) - this would include today's EPSS (Electronic Performance Support System)
2. Self-Teaching Exportable Package - the elearning we know today would fall under this category
3. Formal On-the-Job-Training (OJT)
4. Installation Support School (on or near the employees' workplace) - this would be formal classroom training even though the training may be conducted outside in the field
5. Resident Instruction (away from employees' workplace where travel and living expense would have to be considered) -this would also be formal classroom training.

Just as training designed for the classroom, the other options also need to follow the five phases of ISD to ensure they do what the are supposed to do. For example, even a simple JPA requires:

• Analyzing the various settings and media to determine if it is the most approximate method.
• Designing it so it performs as intended.
• Developing it into a real product.
• Delivering (implementing) it to the workers who need it.
• Evaluating it to ensure it does the job it was intended to do. This also shows the business units that you care about the solutions you deliver (if it ain't worth following up on then it probably ain't worth doing) and you might learn something. Note the evaluation may be as simple as checking with a couple of managers and some of the employees to ensure it is doing what it is supposed to do.

The manual also gives some guidelines for selecting the correct training setting:

• Job Performance Aid
  • close supervision not required
  • task follows a set procedure
  • JPA can be followed while performing the task
  • do not use if:
    • consequence of inadequate performance is high
    • employee lacks prerequisite skill
    • task requires high psychomotor skills
• Self-Teaching Exportable Package
  • close supervision not required
  • task can be self-learned by individual or groups
  • material required for training can be adequate packaged
  • do not use if:
    • task failure would result in injury or damage
    • special facilities or equipment required
• Formal On-the-Job-Training
  • close supervision is required
  • task can be self-learned by individual or groups in the workplace
  • task decay rate is very high
  • do not use if:
    • sufficient equipment is not available for learners to practice on
    • workplace cannot absorb the learners adequately
    • training would be disruptive to normal operations
• Classroom
  • large group must be taught the same thing
  • task difficulty requires a high state of training (task is difficult and requires time to acquire skills)
  • learner interaction is required (such as team training)
  • material required for training cannot economically be placed in the field
  • essential the employee be able to perform upon job entry (high consequence if employees are inadequate performers)
    • do not use if:
    • task may be adequately trained elsewhere

In addition, think blended learning. When I was first trained in ISD we called it BoB (Best of Breed). Blended learning solutions are normally more efficient and effective when designed correctly as they inherit the best of each setting. And do not think of blended as just Brick and Click, but rather any combination of the above, plus more informal options, such as mentoring and social learning media.

Related Posts:

ADDIE Backwards Planning Model

ADDIE and the 5 Rules of Zen

ID is not ISD

One of the trends in the learning industry is proclaiming that a new Instructional Design (ID) model, such as rapid development prototyping, needs to replace Instructional System Design (ISD) because the new model provides more benefits, such as it's newer, dynamic, and faster. Yet ID models differ from ISD models, thus its sort of like saying that a new boat model is going to replace the automobile—yes they are both transportation devices but they do differ in their uses!

ID (Instruction Design) models differ from ISD models in that ISD models have a broad scope and typically divide the instruction design process into five phases (van Merriënboer, 1997):

• Analysis
• Design (sometimes combined with Development)
• Development
• Implementation or Delivery
• Evaluation

Since ISD models cover a broad spectrum they normally do not go into much detail in the design phase. This is where ID models excel. Since they are less broad in nature and mostly focus on design, they normally go into much more detail for the design phase.

Two popular ISD models are ADDIE and The Dick and Carey Model. ISD can also be extended by using Frog Design's model to solve wicked or complex problems as it aligns with ADDIE:

Some popular ID models include Rapid Instructional Design (RID), Gagne's Nine Steps of Instruction, John Keller's ARCS model, Merrill's Component Display Theory, and van Merriënboer's 4C/ID Model.

ISD can be thought more of as a project management tool while ID models are specialized tools used to enhance the learning process.

Omitting ISD and relying strictly on an ID model often omits critical parts of the design process, such as analysis and evaluation. Thus, unless you design for certain groups in an organization or industry in which you know your learners, analysis is important to determine the skill level that the learning program is aimed at. In addition, managers will often identify any performance problem as a training problem, thus the designer needs to ensure it is indeed a training problem rather than a bad process or motivation problem.

Evaluation is not only important to determine if the program is meeting the needs of the organization, but also as a learning tool for the designers themselves.

The best way to use ID models is to plug them into the ISD model as they are needed. For example:

This method allows you to gain the benefit of the ID model that will best suit your needs for enhancing your learning program, while ensuring that your learning program will do what it is supposed to do.

Reference

van Merriënboer, J. J. G. (1997). Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training. Englewood Cliffs, New Jersey: Educational Technology Publications.