Should We Assess Distributed Situation Awareness Before, During Or After Command And Control Activity?

Introduction

NATO [1] describe the battlespace in which command and control (C2) must operate as a ‘problem space’ which is characterized by three dimensions; rate of change, strength of information position and familiarity. The emphasis of NATO member states, and particularly the US and UK service doctrine developments, have in recent years focused on the utilization of agile C2 systems in response to opportunities afforded by technological advances and challenges of modern counter-insurgence warfare [1,2,3,4,5]. The advances of technology and the increased pace of operations means that whilst data is often plentiful, it can be difficult to distinguish relevant information from irrelevant, as mission commanders constantly receive tactical updates [6,7]. To alleviate some of the pressures placed upon mission commanders technology has been applied to aid them in achieving and maintaining Situation Awareness (SA) on the battlefield [8]. SA has been recognized as an important part of performance in land warfare [9]. Understanding SA as part of C2 performance is therefore of interest to the wider military community.

Stanton et al. [10] asserted that “command and control is a collection of functional parts that together form a functioning whole” (p. 11). Team work in C2 systems can be distributed in nature and may involve both human and non-human actors [3,11]. A recent theory of Distributed Situational Awareness (DSA) has been proposed which takes a systems approach to the understanding of SA [11]. Stanton et al. [11] sees SA as emerging from team, or systems, interaction. They argue that this approach “may help to promote a better understanding of technology-mediated interaction in systems” (p. 1288). DSA emerges as a result of information exchanges between parts of the system. DSA is therefore an emergent property which is achieved through interaction or exchange. Such exchanges have been described as transactional SA and provide the means by which DSA is developed and maintained [11,12]. Skyttner [13] argued that “information becomes knowledge only when we decide to put it into use” (p. 207). Skyttner [13] takes much the same position as Stanton et al. [11] who defines DSA as ‘activated knowledge’. Communication therefore plays a key role in the development of DSA in teams [14]. Indeed, Stanton et al. [11] stated that: “it is not possible to have DSA without communication” (p. 1309). They pointed out that the links between agents are more important than the agents themselves in maintaining DSA. Effective team-working depends on information transfer, DSA is therefore concerned with how information is used and distributed among agents in systems [11].

The systems approach may also be influential in highlighting shortcomings of SA in C2 teams; particularly with regards to its role in friendly fire incidents [15,11]. The goal must be to understand and mitigate SA breakdown. Stanton et al. [11] similarly argue that measures of DSA can enable interpretation and comparison of C2 systems. This is supported by Hue [16] who points to the challenge which face the defense community in terms of understanding the characteristics associated with Network Centric Warfare (NCW). By enabling comparison between different C2 structures and assessment of technological innovations, assessments of DSA may have a role in developing NCW capabilities. The ability to understand and influence DSA in C2 systems, however, depends on the availability of data collection methods which are able to assess SA within the particular context of C2 environments.

Whilst research has considered a wide array of measures for team SA (see, for example [17,18]) little light have as of yet been shed on data collection methods for DSA with regards to when they are administered in relation to C2 activity. Modern battlefield environments place considerable demands on C2 teams and the environmental characteristics also impact on the DSA which emerge within the team. This paper therefore poses the question of when DSA should be assessed; before, during or after C2 activity. The review considers three data collection methods and focuses the review to criteria which may be used in qualitative cost-benefit judgments in order to select appropriate measure. Costs are here to be understood in relation to the demands made on the C2 system or team, for instance what sort of access to personnel may be required. Benefits are considered in relation to the output, or the data collected.

Three available data collection methods used to assess DSA are reviewed: the Hierarchical task analysis (HTA), communication analysis and the interview technique called the Critical Decision Method (CDM). Each of these methods have a proven track record when it comes to assessment of DSA and can be applied either before (HTA), during (communication analysis) or after (CDM) C2 activity. The data collected by either measure feed into a network analysis method of assessing SA (e.g. propositional networks or concept maps). This review does not consider data analysis in full and direct the reader to the literature for instruction in the analysis of the data collected. The three measures are here considered with regards to their suitability for use in assessing DSA in C2 environments and were evaluated against fourteen criteria: ability to reveal team interactions, ability to depict the emergence of DSA, level of invasiveness associated with the measure, time to administer, reliability, validity, tools needed, input into design/CADMID cycle, resources and/or training required, access requirements, ability to assess compatible SA, ability to describe SA transactions, discerning between human and technical agents and the theoretical underpinning of the methods. These criteria were developed from the theory of DSA, the characteristics of C2 environments and research methodology. Recommendations are also made in terms of when either method should be utilized. In the following the assessment criteria applied to compare the three DSA data collection methods are considered.

Assessment criteria

Fourteen criteria were applied in considering the appropriateness of the techniques for assessing teams operating in complex C2 environments. These can be broadly grouped into three categories: DSA relevant criteria, C2 relevant criteria and research methodological criteria.

The first category concerns team interaction, emergent DSA, ability to assess compatible SA and ability to describe SA transactions. Team interaction refers to the activities agents perform to coordinate their activities. Emergent DSA refers to the behaviour of the team or system which results from the interactions which takes place. Salmon et al. [16] state that “collaborative systems possess cognitive properties (such as SA) that are higher than individual cognition” (p. 26). Compatible SA refer to the finding [16] that each agent’s SA is different, i.e. not shared, for the same situation. This is due to agents utilising information available in different ways to complete their tasks. SA transactions ensure that agents are aware of the common picture through the updating of individual agent’s SA. SA transactions have been referred to as the glue which holds the system together [12].

The second category concerns: invasiveness, tools needed, time taken to administer and access requirements. Invasiveness refers to the potential impact the data collection process may have on military personnel, tools refer to the material required to execute the method and access refers to required access to military personnel.

The third category concerns: reliability, validity, training and resources required and theoretical underpinnings of the methods. Reliability concerns whether the method can be replicated and give identical results whilst validity refers to whether the method is assessing the right thing (that is, DSA). Training and resource requirements refer to basic instruction into administering the method whilst theoretical underpinning reflects the framework the method sits within. The next section describes the three DSA data collection methods reviewed here.

Dsa measures

The literature offer three data collection approaches that are of relevance to DSA: hierarchical task analysis (HTA), communication analysis and interview methods. HTA has for instance been used to assess SA requirements for the design of systems [19]. Communication, of course, forms an essential part of team collaboration and cooperation [20] and as a result communication analysis has been applied to assess SA in teams (see, for example, [21,20]). Young and Stanton [22] describe interviews as a method for gathering general information which can provide insight into any kind of situation where an individual’s perspective may inform an understanding of that situation. The Critical Decision Method (CDM) [23] sits within the category of interview techniques and has been applied to assess DSA in teams (see, for example, [19,24,25]. In the following each measure is described in more detail.

Before C2 activity—hierarchical task analysis (HTA)

HTA was developed to analyse complex tasks, such as those in the processing industries [28]. HTA analyses goals and operations as the means by which goals are attained, rather than tasks as such [28]. Stanton [11] states that the HTA may be used to analyse systems by considering the goals of the system in detail, however, HTA may also be applied to consider parts of the system, including individual operator’s tasks and those performed by teams. The HTA decomposes complex tasks into a hierarchy of goals, operations and sub-operations or plans [28,27]. This means that the HTA is well equipped to identify areas which require improvement; either training of operators or the design of a system [28]. The measure has been utilised in a range of domains, such as process control, the military, human computer interaction, team skills, training, human error and risk analysis [28].

Salmon et al. [19] for instance utilised HTA to reveal SA requirements to inform the design of systems. They stated that an SA requirements analysis, where all end users SA requirements are comprehensively identified and noted, should begin with a HTA. Data are collected from diverse sources, such as through interviews with subject matter experts (SME), training manuals or other documentation [19]. They go on to explain that following the HTA the relationship between different parts of the system, or team members’, SA requirements can be identified by a graphical representation (e.g. a propositional network/concept map). The aim of these depictions should be to identify: “what it is that needs to be known, how this information is used and what the relationships between the different pieces of information actually are—that is, how they are integrated and used by different users” ([19] p. 216). This means identifying information which underlies DSA and which represents compatible SA (that is, information used in different ways by different team members), what information are transactive SA (that is, information passed between team members) and what information can be both compatible and transactional in use [19]. Salmon et al. [19] recommends consulting SME to complete the last step. Considering DSA, in terms of SA requirements, by assessing the system through a HTA therefore allows system designers to group information meaningfully to support the development of DSA in C2 systems [19].

During C2 activity—communication analysis

It is presumed that effective communications are required for teams to successfully perform their tasks [29]. Weil et al. [20] state that “communication is the choreography of team performance” (p. 277). They go on to argue that the elements of collaboration which aids the emergence of team SA are available in the content of communication between team members [20]. The content of team communication can therefore be observed and measured to gain insight into DSA in operational settings where interviews or other intrusive measures are inappropriate [20]. Communication content (that is, what is said) and communication flow (that is, who is communicating with whom), have been the focus of team research for some time [20]. Several studies have focused on the importance of communication for team SA. For instance, Redden and Blackwell [30] studied radio communications within a squadron which were categorized in terms of critical information based on a framework developed with SME. The data was subsequently analyzed in terms of the extent to which the critical information was present in communication between the squadron members. Galliganl [31] similarly report a study in which communications were modeled to identify areas which benefit, as well as those areas which may be negatively affected, by the introduction of networking technologies in NCW. A further study was presented by Stanton et al. [9] who analyzed communication types and patterns which took place between Brigade level Headquarters and geographically dispersed Battle Group Headquarters. They utilized both voice and digital communications in their analysis of a NCW system to assess the organization’s response to its environment.

After C2 activity—interviews

Klein and Armstrong [32] describe the CDM as a semi-structured interview technique aimed at eliciting knowledge of decision making in naturalistic settings. The CDM “applies a set of cognitive probes to actual non-routine incidents” ([33], p. 464). Klein et al. [33] argues that by allowing respondents to reflect on strategies they used in particular situations, and the decisions they made, a rich source of data can be exploited.

The CDM is most commonly used in face to face interviews; however, this manner of administration requires resources such as access to respondents over longer periods of time. Stanton et al. [18] estimated that between 1–2 hours are required. Given the limitations often placed on access to personnel in organizational settings researchers have adapted the CDM to allow for open-ended questionnaires to be administered, particularly in the military domain [24,25]. Such adaptations are advocated by Klein and Armstrong [32] who argued that development of CDM should be explored to maximize its potential. They suggested changing the execution of the CDM and combining it with other measures. Converting the CDM from a semi-structured interview to an open-ended questionnaire therefore do not breach the integrity of the measure. This added flexibility has enabled application of the CDM to respondents who may otherwise not have been accessible to the traditional administration. In addition to altering the administration of the CDM, Klein and Armstrong [32] also suggest that changes to the probes themselves can be made if the operational environment requires it. It is clear that analysis of recordings of C2 team communications, may support evaluation of DSA during C2 activity. In the following section the three measures are evaluated using the fourteen criteria described above.

Comparison of the measures

As has been established elsewhere (see, for example, [17,20,9,25], DSA can be explored in terms of SA networks which show the knowledge contained by the whole system. SA networks and variations of such networks (such as propositional networks, information networks and concept maps) have therefore been applied as measures of DSA. All of the data collection methods described here (that is, the HTA, communication analysis and the CDM) provide raw data in the form of transcripts which can be used to develop SA networks or any of its variations. The data collection methods are therefore hypothetically equal in the outcome provided—that is, in that each provides a network of relevant concepts or knowledge items. However, the data collection method differs and this difference in collection technique may result in significant differences in the structure of the networks and its content. Such differences can have consequences for our understanding of DSA in C2 teams and for the recommendations regarding technical or organisational designs which are made. Comparing the three data collection methods to consider when DSA should be measured relative to C2 activity is therefore important.

Dsa criteria

The DSA criteria were: interaction, assessment of compatible SA, description of SA transactions, emergent DSA, the ability to considering human versus technical agents and input into design

The HTA enables an identification of agents, both human and technological, role in the system through the sub goal descriptions. These show how the parts of the system must interact to fulfill the goal through executing the plans and completing the task, which in turn triggers further tasks. In this way, Salmon et al. [19] explain that the HTA can show coordination activity of team members as they seek to achieve team goals by identifying the information which will have to be sent, and received, by team members. Similarly, the HTA can show where SA transactions ought to, or must, occur in order to execute plans successfully. By describing the tasks and plans it also becomes possible to show where compatible SA ought to develop between team members. The HTA may show division of labour between human and technical agents and can highlight where technical agents may support the agent. As such the HTA may be beneficial in the concept design phase of the CADMID process. This data collection method is limited, however, by describing the ideal system and cannot take account of what actually takes place within the system or team under study. The HTA may depict emergence of DSA by tracing the triggering of, and execution of, plans to fulfill goals. In so doing the HTA provides an overview of systems level awareness in the form of a graphical depiction such as in a propositional network [19]. The overview of systems level awareness provided may prove incorrect; however, should the system trigger and execute plans other than those anticipated in the HTA, presenting an obvious weakness of the HTA.

The CDM, in turn, can reflect team members’ interaction in that individual team members may refer to a particular colleague, agent or role in their CDM interview. However, where no such references are made there will be no evidence of interaction assessable in the data collected by the CDM. This means that some of the key aspects of DSA could be lost. Without being able to reflect the interaction which takes place in the team, or system, the systems level DSA depicted cannot offer recommendations in terms of support for SA transactions or consider the impact of new technology on teams. The CDM can describe SA transactions, or inferring them, by the references made to significant information and agent utilised during task performance. In other words, an agent who describes how they updated a status report detailing enemy movements and transmitted this to his team has provided his team with an SA transaction. This remains a retrospective description of SA transactions. The retrospective nature of the CDM makes it suited to the demonstration and disposal phase of the CADMID cycle where it can extract DSA relevant data from an already operating system to assess it with a view to modifying the system. In this phase it can also be used to establish knowledge transfer of the aspects of the system which had a negative or positive impact on DSA. The CDM cannot assess technological agents which is a limitation for its input to design and wider system understanding.

An added disadvantage of the CDM arises from the fact that not all personnel may be willing to describe the full extent of what took place during teamwork, for instance if a particular team member failed to pass on vital information or made critical mistakes, other team members may prefer not to “grass” on their colleague. Querying all agents which interacted during a task may remove this limitation and experienced interviewers are able to some extent to navigate sensitive issues and an assurance of anonymity also goes some way to set the conditions for an insightful exchange. The interview condition can, on the other hand, provide just the setting in which someone may feel able to divulge problems which concern them within the team or wider system. The CDM remains vulnerable to the preferences of the individual respondents, however, and so reliable interaction data may not appear in the transcripts.

Although CDM provides an overview of the systems level awareness,it can only provide retrospective insight into DSA. This means that the accumulated knowledge activated during task performance for the team can be gleaned from the knowledge network developed (such as propositional network, information network, and concept map).

In contrast, communication analysis reflects who communicated with whom and in so doing depict the interaction which took place in the team. Indeed, by being able to show the directionality of SA transactions, communication analysis can both consider the flow and pattern of communication as well as the content. This provides a powerful means by which DSA can be assessed and supported in C2 teams. For instance, by considering breakdowns in SA it may be possible to isolate agents or parts of the system that does not interact appropriately, thereby mitigating escalations leading to serious incidents such as friendly fire or accidents. As such, the communication analysis as a data collection method may inform the assessment and demonstration phases of the CADMID cycle. This method can only assess technological agents by showing how technological agents are utilised in a system or team. For instance, a team member may use the radio to communicate or may refer to the GPS verbally in discussions with team members, or if "system logs" are recorded [9, 32]. These references may be utilised in design processes.

If applying the measure of communication analysis it becomes possible to not only provide a systems level depiction of awareness which have emerged retrospectively, but also to trace the way in which DSA emerges over time. For instance by revealing the stages of coordination which the system, or team, went through and show how these stages occurred in conjunction with significant parts of task performance (such as dispatch of resources and critical decisions).

C2 criteria

The C2 criteria were: invasiveness, tools needed, time taken to administer and access requirements. In terms of invasiveness the HTA requires access to SME to verify and inform the descriptions of goals, sub goals and plans. However, the SME may be selected from higher echelons of the organization or may include only one member of the team under scrutiny. Salmon et al. [19] and others (see, for example, [22,11]) advocate the collation of HTA from other teams or systems to prevent replication of similar work. In this way the invasiveness of the HTA may be kept to a minimum. The HTA, by virtue of being completed prior to C2 activity taking place require no input from personnel which may interfere with their task performance. It does, however, require the investment of time in proportion to the complexity of the task and analysis [28]. This means in practice that an HTA may be time intensive, however, the analyst may construct the analysis in such a way that the SME input is minimized, i.e. by consulting material and other known HTA before approaching the SME. The tools needed for a HTA are documents and procedures as well as observation of tasks being executed or similar “show and tell” exercises performed with SME. Access in terms of collecting DSA data by the HTA method can be limited to a small number of SME (as few as just one person) who need not be operationally involved.

Communication analysis require minimal invasion where communication can be recorded. Both audio and textual communication may be recorded and later transcribed for analysis. Whilst some team members may be distracted by knowing that their communications are recorded in many instances this already occurs for safety reasons (that is, for use in case of accident investigations and for training purposes). Research have shown that individuals become accustomed to being observed, either through direct observation, video-recording or audio-recording, that they continue as if they were not observed [34]. Therefore it can be expected that in a relatively short period of time the recording of communications should not lead to undue distraction of personnel. However, to be successful the communication analysis method requires access to all communication which takes place between team members. This means that any radio communication and any face to face communication should be recorded. This data collection method require little administration time during task performance, however, it require preparation (such as set up of recording equipment and decision when and where to record activity) and in transcribing the recorded data. Resources required are standard PC with word processing facilities, addition resources such as transcription software may be of benefit but is not essential.

The CDM, on the other hand, requires access to personnel after an event and preferably to all personnel from all areas of the system for a face-to-face interview and the measure cannot adequately consider technological agents, as such this method is both invasive and place high demands on access. In C2 environments personnel are rarely inactive which may limit the times at which interview may take place. The longer the delay between task completion and the interview, the greater the chance of memory degradation [18]. Further limitations of the technique are the cognitive probes of which many are not relevant to DSA.

Most face-to-face interviews take between 1-2 hours, as does the CDM [18]. The use of an open-ended questionnaire would perhaps reduce the time taken to administer somewhat though not much less than an hour. Where an online open-ended survey has been developed, as described in the introduction, access may improve and the level of intrusiveness could be reduced. Amendments may also be warranted to rephrase probes to ensure relevance to DSA. If meaningful data are to be gained, however, the response time by personnel would still have to be between 40–60 minutes. Pen, paper, and recording devices are tools which may be needed if the method is conducted as an interview. Where the method is utilized as an open-ended survey these may be done using either online survey tools or printed versions.

Research methodological criteria

The research methodological criteria were: reliability, validity, training and resources required, and theoretical underpinnings. The HTA is associated with low levels of reliability but with high levels of validity. As a data collection method it is related to cognitive task analysis method. It requires time intensive training and practice to be conducted well and practice in making decisions to end the development of an HTA is important as this must occur at the right level of detail. The CDM method is associated with low levels of reliability and its validity is also questionable due to the probes which are currently not relevant for DSA. The method also requires that significant time is devoted to training and practice to elicit the richest possible data. Communication analysis is also associated with low levels of reliability but with high levels of validity. No training is required for the administration of the communication analysis method, however, instruction is required to ensure that high quality transcripts are developed (such as how the meaning of words may be retained when taken out of a spoken context).

Table 1 shows a summary of the comparison of the three DSA data collection methods against the DSA criteria, Table 2 shows a summary of the comparison against the C2 criteria, whilst Table 3 shows a summary of the comparison against the research methodological criteria.

Discussion

The HTA, communication analysis and CDM have been used with success to depict DSA in areas such as civil energy domain [18] and the military domain [see, for example 19, 24, 25]. However, the suitability of these methods for the C2 environment has not been considered in detail. The aim of this review was therefore to compare the data collection methods on fourteen criteria to highlight the relative advantages and disadvantages of each measure for the challenges which faces teams operating in the C2 domain. It was asserted that given the highly changing and information rich problem space which characterizes modern battlefields [1] data collected of DSA must be able to reveal the interactions which take place between team members, depict the emergence of DSA, whilst being non-intrusive and as time efficient as possible. The methods available to assess DSA, in addition, lend themselves to assessment at different stages of C2 activity, with the HTA enabling assessment before, the communication analysis during and the CDM after such activity. The selection of appropriate data collection method must therefore take into account not only the criteria relevant to the C2 domain but also the stage of C2 operational performance at which the method may be applied with the relative output the method can offer.

This review has shown that the HTA, which can be applied before C2 activity takes place, may highlight the areas where interaction ought to take place for optimal team performance and development of DSA. Salmon et al. [19] pointed out that this has the added benefit of highlighting areas where technology may be utilized to support SA transactions within the system or team Communication analysis, by virtue of recording teamwork during task performance, affords a real-time depiction of DSA as it emerges through team interactions. The ability of the communication analysis, such as recorded in communication logs, to reflect emergent DSA within C2 teams makes it a powerful tool for assessment in C2 environments. The CDM, on the other hand, provides an “after the fact” image of C2 teams’ SA. In other words, the CDM shows the DSA which did emerge for a team or system, rather than provide a tracing of DSA as it emerges.

This latter quality, the tracing of DSA emergence, is one which is of particularly relevance given the high pace of change and the distributed, decentralized and networked qualities which characterizes modern C2 environments. Where changes occur rapidly it is vital that one can outline the adaptations being made within the team and the resulting impact this has on the DSA which is developed.

When considering team interactions the HTA ensures that the goals which are interdependent can be highlighted in advance of the activity. This means that the HTA may serve as a training tool for increasing the awareness of team members, in advance of operations, of areas where they must fulfill coordinating roles. The HTA may also serve as a check against which performance can be assessed in terms of whether the team was coordinated in the required manner. It may also serve as a means by which weaknesses in the system can be highlighted and technological support may be directed. Conducting a communication analysis during C2 activity has the unique benefit of being able to reveal the important SA transactions which occur during teamwork. With this method it is possible both to consider the frequency of communications between team members and the pattern of communication associated with a team. Scrutiny of frequency of communication and patterns of interaction as advocated by Jentsch and Bowers [29] can reveal areas where technology may support DSA in C2 teams, or it can be applied to assess the impact of new technology.

As such by using communication analysis it becomes possible to consider the role communication plays in the development of DSA both in terms of good and inadequately developed awareness [14]. Hence, communication analysis enables an identification of the links between agents as advocated by Stanton et al. [11]. This in turn enables a comparison of the relative performance of C2 structures [11]. The CDM may reveal the number of times individual respondents refer to specific team members or agents within larger C2 system; however, it cannot demonstrate objectively how the team members interacted to solve the tasks.

The importance of showing how teams exchange SA transaction and interact to enable DSA to emerge is particularly acute for the C2 domain where SA breakdowns may lead to catastrophic consequences. The output of the measures should therefore be used to mitigate SA breakdown and increase support for the development and maintenance of DSA within the team and the C2 system. By assessing DSA at the beginning of C2 activity it may be possible to influence battlefield technology design by specifying what functions the technology must have and how these should be allocated for optimal achievement of DSA, The output of the data collection achieved with the HTA, for instance, may be usefully applied to inform design at the concept phase of the CADMID cycle. By assessing DSA during C2 activity data collected may inform acquisition decisions concerning use of existing technology to best support the system, and by assessing DSA. The communication analysis lends itself to collect data that may be used in the assessment and demonstration phases of the CADMID cycle. Whilst, by collecting DSA data after C2 activity, it may be possible to inform future operational use of battlefield technologies to support DSA. This can be done by the use of a retrospective data collection method, such as the CDM, which may feed into the demonstration and disposal phases of the CADMID cycle. In this way the data collection methods are not only linked with the stages of C2 activity but can be related to parts of the CADMID cycle.

Military personnel are by the nature of the operations they perform mostly inaccessible. Rarely can personnel be spared for lengthy discussions on the goals of their activities or for face-to-face interviews; in addition interruption of performance during operations could have dire consequences. As such, any data collection methods applied to assess DSA must be non-invasive and time efficient. The HTA could potentially be quite invasive by engaging all team members in informing the hierarchical development of the goals, sub goals and plans. However, the analysis can be constrained to include only one SME. Additionally, as the analysis takes place before C2 activity it can limit the intrusion considerably. Where communication logs may be recorded this method present the least intrusive option, compared to the CDM, as such recording can capture communication which takes place naturally within the C2 team. This also renders the communication logs as the least time intensive measure as it does not require the use of personnel time directly. This review has considered three measures of DSA specifically for the C2 domain taking into account its particular challenges in comparing the methods against fourteen criteria. Each method on their own has proven useful as data collection tools for DSA in the military domain [11,19,22]. Whilst it is relevant to discuss the methods separately it should be noted that where possible combining the methods may provide the most comprehensive results. In this way the HTA may set out what ought to be achieved, the communication analysis may consider what takes place whilst the CDM can allow personnel to reflect on what took place.

Table 2. summary of the comparison of the three methods against the C2 criteria.

Table 3. summary of the comparison of the three methods against the research methodology criteria.

As such, if intrusion and time demands are less critical, for instance during training exercises, combining the CDM with communication analysis would give the added benefit of the reflections of the personnel on their and team members actions. Whilst, if the aim of the analysis is to consider where technology may best support C2 teams coordination activities to mitigate SA breakdown a combination of the HTA and communication analysis may be preferred. Considering each of the three measures against all fourteen criteria overall it becomes clear that where only one data collection method is feasible the use of the communication analysis method would give the greatest advantages. This is due to the methods ability to input into larger parts of the CADMID cycle, its potential to allow real time tracing of team interaction and SA transaction, and by extension revealing how DSA emerges over time. In addition this method is associated with the least impact on military personnel despite requiring access to communication and high demands on the staff who must transcribe the material.

This review has shown that the HTA reveals the areas of interaction and emergence of DSA which are latent in a system and may highlight areas in need of support or improvement through system design. Communication analysis, on the other hand, reveal the teams DSA as it emerges and enables a comparison between C2 structures as suggested by Stanton et al. [11]. The CDM in turn enables a retrospective insight into the overall systems awareness which emerged and can provide important insights into relevant personnel’s reflection on their performance. Assessment of DSA in C2 teams remain an important area for researchers and practitioners as either measure may inform technology development, selection of C2 structures, training and doctrine, as advocated by NATO [1] and DCDC [2].

Conclusion

SA has been established as a key part of C2 performance, in particular the role of SA breakdown in human error and fratricide has led to an increased interest in the phenomenon. This paper has presented a review of three measures for assessing DSA in the C2 domain: the HTA, communication analysis and CDM. It was asserted here that measuring DSA in C2 environments require unique attention as the ability to understand weaknesses of C2 teams’ development of DSA can influence the adoption of technology and training of such teams to improve battlefield performance. C2 teams require efficient information sharing and interaction to achieve DSA, team interaction is therefore a vital aspect of both DSA and C2. As such, measures of DSA must enable a representation of the interactions which takes place within the team and between human and technological agents. The HTA was shown to be able to provide an overview of the interconnectedness of goals in the team and as such may highlight areas where teams may have compatible SA and where SA transactions are likely to take place. The HTA can therefore both be useful to inform system design and as a check against C2 teams performance. The communication analysis similarly has a significant advantage in that records of communication can highlight areas where technology and training may be required to maximize the C2 structure’s potential by reflecting frequencies and patterns of communication between team members. Further research should consider the utility of each of the three measures on their own as well as in combination in order to assess all aspects of C2 activity.

References

Authors

Ms Linda J. Sorensen is a doctoral student in Human Factors at the Faculty of Engineering and Environment at the University of Southampton with a research interest in military human factors. Sorensen may be contacted at ljs1v09@soton.ac.uk.

Professor Neville Stanton, PhD, holds the chair in Human Factors in the Faculty of Engineering and the Environment at the University of Southampton, UK. He has published over 20 books and 160 journal papers on Human Factors and is consulted by commercial and defence companies throughout the world. Professor Stanton may be contacted at n.stanton@soton.ac.uk.