Decision Making / 3 / 3.9

3.9 OBSERVATIONS ABOUT THE USE OF CONCEPT MAPPING AS AN ANALYTICAL TOOL

Maps can contain large numbers of concepts if mapping is done in a comprehensive way. Despite this being done during the research activities underlying this book. It is rarely necessary to build highly detailed maps. The approach described in this book has been chosen to reveal the true nature of those problems being addressed. Mostly we need to build maps containing relatively few concepts, but those maps must be highly incisive.

When using any from of causality-directed graph, selecting the appropriate level of aggregation is important. This is a matter of skill and experience. Coyle (1996) and Coyle and Exelby (2000) describe a ‘Cone of Influence Diagrams’ within which diagrams drawn at higher or levels of aggregation fit. The idea of reducing the level of complexity being described to a point where clarification becomes possible is an overall purpose in diagramming (Craig, 2000).

Table 3-1. Major Risk Key Performance Indicators—Black Hawk Case

Clarification follows recognition and recognition is context-dependent. To ensure we create enhanced opportunities for executive decision-makers to recognise patterns of behaviour we have to create the right context (Richardson and Pugh, 1981; Carroll and Johnson, 1990; Klein et al., 1993; Klein, 1998; Gigerenzer, et al., 1999). Consequently, we initially focus attention on setting and developing the context. Facilitating ‘situation awareness’ (Klein, 1998) is critically dependent on recognition. It is also an essential precursor to understanding, understanding which must be developed before we can develop workable management strategies. The main threat to any mapping activity is in failure to be objective. This failure could result in the building of maps representing what the person building the map hoped the author or interviewee might be saying rather than what that person actually said or wrote. The mapper cannot let his own views or biases to be built into the maps. One technique for minimising bias is to present the complete, or partially complete, map back to the author for validation. This is consistent with an action research/action learning approach (Revans, 1982; Perry and Zuber-Skerrit, 1991). The knowledge-elicitation technique used during empirical case study research upon which this book is based involved a two-staged interview process:

• During the initial interview, a series of questions were posed to the interviewee. The interview was recorded. During the interview, concepts were noted and linked as far as possible. This led to the building of a set of partially complete maps. After the interview, a transcript was prepared from the recording. The transcript provided the additional information needed to build complete maps of what was discussed during the interview. Occasionally maps were found to be mildly confused, illogical or incomplete. Sometimes, there were omissions from logical arguments. This was due, in part, to lack of clarity of expression on the part of the interviewee. It was also due to errors of interpretation by the interviewer during the mapping.
• When the maps were presented back to the interviewee at a subsequent meeting, the interviewee was briefed on what the map portrayed. The interviewee was encouraged to ask questions regarding the map, as it was presented, and to make corrections. Up to this point, the scope was limited to the original focus of the initial interview. Only when all corrections were made, was the interviewee asked if he had any afterthoughts that might be included as additions to the map.

When mapping, particular attention must be paid to:

• Inter-coder reliability. Identifying concepts from documents, requires skill in parsing, a good understanding of the written language and an awareness of difficulties introduced by linguistics. Threats to inter-coder reliability, the fidelity of the reader identifying exactly what the writer intended, are ever-present in such tasks. The level of inter-coder reliability achieved here could not be assessed because there was no method of referring the map directly back to the originator. Despite this, the overall error introduced is considered low. For example, Figure 3-6 Context Diagram Focusing on Concept No. 19— failure to learn, identifies some 34 links to the central concept, and even if a significant number of linked concepts were incorrectly attributed, a large number of links would remain. Concept No. 19 failure to learn is most important: this is intuitively obvious, and the Coroner’s meaning was clear in this regard. Further to the two-staged interview process described above, it is discussed in subsequent chapters, that a slight methodological change was possible because the client group, the originators of the map, could be continuously engaged. In those cases, inter-coder reliability could be maximised by processes of validating maps directly with the originators, and using the template described at Table 6-1.
• Identification of important nodes. Further to the methodology introduced in Chapter 1, methods for validating causal linkages and validating maps are explained at Section 6.7.3. Identifying important nodes involves more than identifying the number of connected nodes; an assessment of the strength of influence must be made at each stage of map development. A good test is to gauge the effect on the node under consideration that is produced by the trial removal of selected linked nodes. Where the effect is small, the connected node may be considered unimportant. Similarly, if removal of a specific node destroys the logic or minimises the influence, then that node is important. This also assists in determination of what to include within the map.
• Where and which way to draw links. It is not uncommon for different individuals or groups to produce maps that vary, even when presented with the same information. This is because each of us has different ways of interpreting our world experiences. Kelly (1955) identified this in his original theory of personal constructs. What is important is that maps produced, either as individual (cognitive) maps or group (concept) maps do not contain ambiguities. Doubt about where and which way to draw links in a concept map, can be minimised by following the procedure described at Section 6.7.3 and using the template described at Table 6-1.
• Objectiveness of concept maps. Application of a two-stage process in which maps are validated during a follow-up meeting is intended to ensure maps developed as a result of interviews faithfully reproduce the views of the interviewees. The processes described in the tutorial at Chapter 6 are designed with objectivity in mind. When developing concepts from documents, as was done here at Chapters 2 and 3, the same mechanisms, which would otherwise to minimise the introduction of biases, do not exist. Consequently, the maps produced are the sole interpretation of the person who develops the map and the interpretative skill of that individual.