Evaluate and Adaptation

This is the fifth post in a series on extending ADDIE in order to solve wicked or complex problems:

• Post 1: Innovating With ADDIE
• Post 2: Analysis and Immersion
• Post 3: Design and Convergence
• Post 4: Development and Divergence

Evaluation in ADDIE is normally composed of two parts:

• Formative Evaluations: a method of judging the worth of a program while the program activities are forming in order to make on-the-spot corrections.
• Summative Evaluations: a method of judging the worth of a program at the end of its activities (summation), with the focus being on the outcome.

In addition, perhaps the most popular methodology for evaluations is Donald Kirkpatrick's Four Level Evaluation Model.

Frog Design's Adaptation is less of a formal approach and more of a causal approach — "stepping back and looking for new directions to go." Wicked or complex problems often have no clear directions when it comes to improving the initial solution, thus you simply take another crack at it to see if you can add new or better functionalities. Evaluation and Adaptation could be scaled as:

The point that you wound up on the first continuum of Analysis and Immersion will more than likely determine the point that you should be on this continuum. That is, if you are working on a simple to complex problem, then you will normally use an Evaluation technique to check your initial solution (left side of continuum). However, if you are working on a complicated to complex problem (wicked problem), then you will normally be to the right of the scale and thus use Adaptation to check your solution.

Next up, final wrap-up on innovating with ADDIE by using Immersion, Convergence, Divergence, and Adaptation.

Development and Divergence

This is the fourth post in a series on extending ADDIE in order to solve wicked or complex problems:

• Post 1: Innovating With ADDIE
• Post 2: Analysis and Immersion
• Post 3: Design and Convergence

Development in ADDIE is typically thought of as creating the learning content, products, and services, in addition to selecting the media that will carry or deliver them to the learners. Frog Design defines Divergence as branching out beyond what is normally does in its ecosystem in order to take in a wider footprint that provides a holistic solution:

The Development and Divergence Continuum works differently from both the other two ones:

• In the Analysis and Immersion Continuum the complexity of the environment determines at what point you are on the scale.
• In the Design and Convergence Continuum the goal is to aim for the center of the scale.

While the goal in the Development and Divergence Continuum is to aim for both ends — combine “what works” with “what will make it better."

For example, blended learning is normally considered a combination of elearning and classroom learning. eLearning allows the learners to learn at their own pace, while the classroom portion provides the needed social engagement. The Army has found this to be a superior form of learning and is now moving to a dL (distributed Learning) environment. Note that the “d” is not capitalized as only the “Learning” is emphasized — classroom learning is used when it makes sense.

Another example is the greater consideration of informal and social media when creating formal learning platforms.

Thus, rather than just working from just one point in the scale, Development can often be improved by working from two or more points in the scale.

Up next — Evaluate and Adaptation.

Design and Convergence

This is the third post of a series on extending ADDIE in order to solve wicked or complex problems:

• Post 1: Innovating With ADDIE
• Post 2: Immersion and Analysis

In the last post I discussed using a mixture of Analysis and Immersion in order to gain insight and understand the shape of the problem. The next step in Frog Design's framework for solving problems is Convergence — the bringing together of everything you learned in the first step in order to integrate things together; or in other words — to create a whole.

Now this is basically what you do with the Design phase of ADDIE, except with ADDIE, it implies a set of steps, such as developing learning objectives, identify the learning steps, developing tests, etc. If we look at the two terms being placed on a continuum it would look like this:

 

Unlike the Immersion and Analysis Continuum in which the complexity of the environment determines where you would be placed on the scale, the goal here is to aim towards the middle. That is, if you follow the design process too closely, then you might be turning it into a process model as discussed in the ADDIE Timeline that may take all the creativity out of your learning platform; in addition, it may be moving too far away from Agile Learning Design in which 1) individuals and interactions come before processes and 2) you need to respond to change rather than just following a plan.

And if you go too far to the right side of the continuum then you may end up loosing focus of the goals and objectives of the learning platform. Thus, it is a balancing act of determining the correct level of process (design) with the the right of amount of creative freedom (convergence). If you find yourself at one of the extreme ends of the continuum then you need to seriously question if this is really the correct level of both design and convergence that is required for the project you are working on.

One method that will help you from straying too far to the right or left is to use a mapping or graphic technique, such as Cathy Moore's Action Mapping:

Design Lively Elearning with Action Mapping

The business of design is to make things that people really want: useful, usable, and desirable — Shelley Evenson, associate professor of interaction design at Carnegie Mellon University's School of Design.

Bill Moggridge (2007) wrote that iterative prototyping, understanding people, and synthesis are the core skills of design and that these skills are central to design:

• Iterative Prototyping: successive small-scale tests on variations of a limited function prototype in order to permit continual design refinements.
• Understanding People: having a basic foundation of the cognitive and behavioral sciences.
• Synthesis: applying prior knowledge and skills to produce a new, innovative, or original whole.

The Design in ADDIE and the Convergence in the Framework for Solving Problems may be thought of as a combination of Shelley Evenson's quote and Bill Moggridge's design fundamentals in order to create a holistic solution.

Up next, the Continuum of Divergence and Development.

Reference

Moggridge, B. (2007). Designing Interactions. Cambridge, Massachusetts: The MIT Press.

The Immersion and Analysis Continuum in Simple to Complex Environments

In my last post I discussed Innovating With ADDIE by using the X Problem Solving Framework of Immersion, Convergence, Divergence, and Adaptation in conjunction with ADDIE. In this post I'm going to elaborate on the first step of the framework — Immersion and Analysis:

When presented with performance or learning challenges there is a continuum on how to approach the problem that is composed of analysis on one end of the scale and immersion on the other end.

Allison Rossett & Kendra Sheldon defined Analysis as “the study we do in order to figure things out” (Beyond the Podium: Delivering Training and Performance to a Digital World ). This term is normally associated with the traditional steps of analysis, such as performing a needs assessment, task analysis, and building performance measures.

While on the other end of the continuum is immersion. It is defined by Adam Richardson of Frog Design as “the method of soaking yourself in understanding what the problem is that you are facing in addition to informing the other three parts of the framework.” This can be thought of jumping into the problem in order to understand it, but not being able to use traditional analysis tools or instruments due to the complexity of the problem.

In most learning or performance design situations we don't use one or the other but rather a mixture of the two, thus it is a Sliding Scale Continuum:

For example:

Left of Continuum

Simple problems will be composed mostly of analysis because we have Exemplary Performers that we can use for models. While this is normally one of the easier learning platforms to design, it does have a couple of pitfalls. The first is thinking that since it is fairly easy to design, it is also easy to learn and perform, thus we fail to build practice time into the learning platform. Designers often become so absorbed in their work that they fail to realize how much time they are putting into it, thus they spend weeks working on the task, then think they can transform it into a 30 minute information dump. The second pitfall is failing to support the informal learning that must occur after the formal learning because there is an average of a 1:4 ratio in which one hour of formal learning produces four-hours of informal learning.

Middle of Continuum

As we move to more complicated design environments there is no Exemplary Performers to be found, thus we have to rely on our own understanding of the problem and other experts who have knowledge of the type of problem you are trying to solve. Thus we are moving towards the middle of the continuum:

• We are doing analysis in that we are interviewing experts to create a best practice.
• In addition, we are immersing ourselves into the environment of those most affected by the learning solution in order to complete our understanding of the problem that the experts cannot elaborate on.

Like the simple environment, you will have to watch for the two pitfalls; in addition, in this complicated learning environment you will normally perform more sets of iterations to ensure you get the feedback of those affected.

Right of Continuum

As we move farther to the right on the scale to complex learning design environments there are no Exemplary Performers and few or no experts to draw upon, thus we get most of our information by totally immersing ourselves into the environment of those most affected by the problem in order to “paint a picture of an emergent practice.” This is because as Rittel (1972) discovered — the best experts within these types of environments are those affected by the solution — since they are the only ones to have experienced the complexity of the problem, they are the best experts for helping to improve that environment.

You need to look for solutions that support informal learning. This is because the complexity of such environments normally needs a small seed of formal learning with large nourishments of informal learning.

Performance Model

In addition, a performance model (such as the one shown below and I discussed in this post) acts differently in the continuum of analysis and immersion.

That is, when you are presented with a problem that is mostly on the analysis side of the continuum, the type of performance problem will normally become apparent within a short time and it will be more of a simple nature. However, as the problem starts to move to the immersion side of the continuum, you will find yourself having to dig deeper and longer to discover the true nature of the problem. In addition, rather than being a simple problem, it will normally be composed of several parts that require multiple solutions, such as fixing a process, creating a learning environment, and redesigning work stations.

Next

Thus, when presented with performance problems we have to think along the scale or continuum of Immersion and Analysis, depending upon the complexity of the problem.

In my next post we will discuss the Sliding Scale Continuum of Planning that is composed of Design on one end and Convergence on the other end.

Reference

Rittel, H. (1972). On the planning crisis: Systems analysis of the “first and second generation.” Bedriftsokonomen. No. 8, pp.390-396.

Innovating With ADDIE

Every few years the theme of “ADDIE must die” pops up on the radar. This would be a good thing except there are always two major flaws in the argument:

• Instructional Designers are misinformed about what ADDIE is — they refuse to believe that the development of a model such as ADDIE is also iterative in nature — their arguments are always about the first iteration of ADDIE. It almost seems as if they want to stick with the practice of waterfall methods.
• They offer no suitable replacement.

To combat the first flaw I have written several posts on ADDIE. I recently combined these posts into an ADDIE timeline with a discussion on the timeline.

I don't have an answer for the second flaw, although I did write a series of articles on Agile Learning Design that can be used to extend ADDIE.

Which brings me to the purpose of this post — I recently listened to an outstanding podcast, Got X Problems? by Frog Design in which they discuss innovation and why are so few organizations are seeing results. In this episode they discuss solving wicked or complex problems from 21st century challenges that defy conventional planning. What struck me about their method is that it almost perfectly aligns with ADDIE, except that you have to approach it from a slightly different mind-set. In the podcast they use a four-step process:

• Immersion — soaking yourself in the problem to harvest customer insights
• Convergence — bringing together all things such as physical, technology, software, and services into a logical design
• Divergence — exploring new advantages
• Adaptation — stay nimble in a fast-moving environment by going in new directions when facing roadblocks — based on learnings

These four mind-sets would map with ADDIE as:

If you need a different mind-set when facing difficult challenges then I encourage you to listen to the podcast. The RSS feed is http://feeds.frogdesign.com/frog-design-blog/design-mind-on-air. The episode is Got X Problems, dated March 15, 2010. The podcast is about 15 minutes long.

Team Leadership Model

A model for implementing Team Leadership: Team Leadership Model