12.22.2008

Capture the Performance

Once the impact or desired result has been defined, the next step in the backwards planning process is to define the performance level that will support the impact. That is, what exactly must the performers do to ensure that the results or outcome is achieved? Defining the desired performance by recording the steps gives the designers an outline on which to base the learning process.

Defining the performance is normally achieved through two means: explicitly and expository. Explicit performances are ones that can be readily observed because they are already in place. For example, a manager who wants to ensure that new hires are trained in a process that is already in place or a manager who has observed an expert performer master a process and now wants to ensure that the rest of her department models that performance.

Expository performances are new, thus there are no exemplary performers to observe in order to base the performance steps on. Rather, the analyst has to prompt an expert to interpret and describe something based upon her background experiences and knowledge of the subject or task. For example, when a new process is put into place or a process that has been radically changed to produce a more efficient and/or effective one. In addition, tasks that are normally associated with "knowledge workers" often have no clearly observable actions and can often be accomplished in multiple ways, thus a definable performance must be built.

It should be noted that some performances will not be exclusive to one or the other, but rather a combination of the two, however they will normally fit more readily in one of the two performances.

Defining the performance is normally achieved through one or a combination of two means: observing and interviewing Exemplary Performers (EP) and/or Subject Matter Experts (SME).

We can thank John Howe and Dave Ferguson for the term, "Exemplary Performer" (exemplar) - a practitioner who currently does the job, produces exemplary results, and who's widely seen as outstanding in those two dimensions.

Now I'm going to differ with Dave and John on the term of Subject Matter Expert, mostly because I held the title of SME for over a year while serving in the Armed forces who basically define a SME as an expert in the job who works with an instructional designer in the development of a learning/performance process. And for the purpose of this discussion I'm going to add: the expert may not currently perform the job, but her experience and knowledge about the job has given her insights into the performance, especially expository performances.

Explicit Performance

This type of performance is normally easier to define and break down into tasks because you have a model of excellence to observe and interview. Observations are normally accomplished through:
  • Observation Task Analysis: Observe the tasks required in the performance under actual working conditions and record each step for performing a task and the standards of performance.
  • Simulated Task Analysis: Observe the tasks required in the performance using simulated working conditions. The working conditions should match the job environment as closely as possible. Record each step and standards of performance in addition to receiving input from the performer when the simulation does not match actual working conditions.
In addition to the observations, an "Interview Analysis" is also performed: consulting with the exemplar to determine the required steps and standards of performance. This is normally used to validate the data gathered by the observations. Note that this method should not be used alone as performers often leave out vital steps as they have performed the task so often that some of the steps become internalized, thus they fail to acknowledge in even doing so.

In addition, observations normally only include the what and how, thus the interview is used to gather the whys. For example, Icihro always goes through the same basic routine when he approaches the plate to hit.

Icihro

Part of the routine is a tug on the shoulder of his uniform. While some of his hitting techniques can easily be understood by observation, this part is not readily explainable:

  • Is a habit?
  • Is it a lucky superstition?
  • Is it part of a routine to put him into a zen-like hitting state?
  • Does it loosen his sleave so as not to restrict arm movement?
Questions are asked to determine if this part of the exemplar's performance is required. If it is, such a loosening his sleeve for full arm movement, then the "why" is important so that it can be explained to the learners. Perhaps it might be to put him in a zen-like hitting state and if so, do new performers need to do it or can they learn it on their own or should it be put into an advance learning process once they have learned the basics?

Expository Performances

Unlike explicit performances, expository performances have no exemplars to observe and interview, thus the performance steps are normally obtained through:
  • Content Analysis: Analyzing operating or technical manuals (to include web resources) in order to determine the steps and standards of performance.
  • Interview Analysis: Consulting with a SME to determine the required steps and standards of performance.
While explicit performances normally rely mostly on exemplars to model the performance, expository performances rely more on a network of content, SMEs and EPs to first build the model and then to test the model. In addition, you have to work more closely with your customers to ensure the model will work for them (buy-off).

The Office

As you work your way through this network, you will often discover that there are a number of ways to accomplish a task within the performance. A simple example is a software program in which you can use hot-keys, a context menu (click the right key on the mouse), or the main menu at the top of the screen. Thus you have to decide if all three methods are actually needed or if you can reduce the number for more effective and efficient learning. For example, you might decide to concentrate on the context menu whenever possible. However, since the performers will use the copy and paste command quite frequently, then the hot keys for these two commands will also be taught.

Capturing the performance provides the designer with a blueprint for the next step in backwards planning -- planning the learning process.

3 comments:

Anonymous said...

Don, I should make clear that my friend John Howe's use of 'exemplary performer' tends to be specifically in areas where it's difficult to tease out the tacit knowledge and the processes used by such people in (for want of a better term) knowledge work. John's almost ascetic in his belief that outsiders -- including instructional designers -- tend to impose their own structures and perceptions onto a work setting that they don't (and can't) know.

I'm a bit more laissez-faire than John, and have often worked with subject-matter experts, though I do have a bugbear about calling them "SMEs," a term that here in the States often rhymes with "spree." Certainly someone like your armed forces self could have a great deal of knowledge about the task.

I do think that what you call expository performance, which I'm reading as tacit knowledge put to use, can more easily go out of date for a subject-matter expert. That's a potential risk, not a bar to its use. (There's a difference between a relevant example and a war story; the latter has far more useless detail, and often is told in first-person-singular.)

In some of the work I did with John Howe, exemplary performers at a federal agency identified some critical skills for people new to a job overseeing large project carried out by contractors. Discussions among the exemplars helped make explicit variables to consider for a given decision, and some ways the exemplars calibrated the weight of those variables. In some ways, there was a lot of "it depends." The value of the exercise was in making those considerations explicit, and then modifying actual situations to create realistic case studies for the newcomers to tackle.

Another outcome was a sort of process map or decision guide, a flowchart highlighting some of those considerations and questions the exemplars found worthwhile to ask. In one sense, this was organization imposed by outside designers (John and me); however, the exemplars reviewed and revised both the decision points and the acceptable outcomes.

Particularly for newcomes in a knowledge-work area, I think there's a kind of apprenticeship in which you work on seemingly straightforward problems. As you do so, you become aware of nuances that might simply have distracted or overwhelmed you earlier on.

Donald Clark said...

Dave, thank you for the insights. I have also heard other instructional designers comment on their disdain when hearing S.M.E. pronounced as an acronym -- "smee." Having held the job title, such use never bothered me. Since this was a large training organization, there were also a few other SMEs and I never heard them complain if someone used it as a word.

I'm wondering if this is just an instructional designer thing or have times changed and SMEs nowadays also look at the word "smee" with some disdain? I'm also wondering what will happen if "exemplary performer" (E.P.) becomes a common term and we start referring to them as "epps"?

Anonymous said...

Don, in my case, it's more a personal quirk than disdain (though I can manage that quite easily for many things). I'd agree that in large organizations with significant training development, SME (as letters or as a word) becomes an accepted designation.

It's distance that puts the "jar" in "jargon," I think.