Team And Collective Training Needs Analysis (TCTNA): Identifying Training Requirements And Specifying Solutions

Introduction

Military capability is typically delivered by force elements at the team and collective (team of teams) level, such as from infantry sections up to brigades and above. Team and collective training provides the opportunity for individuals to practice together to create effective force elements. Significant features of such training are its scale and complexity. As an illustration, training an armoured battle group involves a training audience which may be a thousand strong and requires significant amounts of live or virtual real estate, a plethora of systems that the training audience are operating, and considerable numbers of training and supporting staff (possibly a hundred or more) to observe performance, control the environment, provide opposing forces, capture data and provide feedback. Furthermore, the instructional team requires extensive supporting facilities to carry out their tasks, such as all terrain vehicles for observer controllers to follow the elements of the battle group as they manoeuvre and a sophisticated monitoring system, supported by a team of analysis, for the set of weapons effects systems used for all the direct and indirect weapons.

The team and collective training challenge is exacerbated by the ever-increasing complexity of contemporary warfare, characterised by Lt Gen Newton (Commander Force Development and Training, British Army) in his opening address to the International Training and Education Conference 2010 as a “wicked problem”. At the same time, budgetary and environmental pressures are leading to the reduction in viability of live training and necessitate increasing reliance to be placed on synthetic training environments. Consequently, it is imperative that training environments are correctly specified so that effective training environment options can be developed which fully exploit the benefits of simulation without overlooking the key attributes of live environments that determine where live training is essential.

The current analytical process that is used within UK MoD and other NATO nations to determine training requirements, specify training methods and training media/environments, and identify appropriate training options to meet these requirements is referred to as Training Needs Analysis (TNA) [1]. (In the US the equivalent process is termed Front End Analysis). The analytical techniques for conducting this analytical process are well understood in the individual training context, and standard texts on instructional design [2,3] and extant military publications such as the current MoD TNA guide [1] provide guidance on appropriate techniques. However, whilst there is a significant body of research on team performance, including notably the outputs of the Tactical Decision Making Under Stress programme funded by the US Navy [4], relatively little appears to have been produced in terms of techniques to support TNA for complex team and collective training. In 1998 Baker, Salas and Cannon-Bowers [5] observed that the critical area of team task analysis had not been adequately addressed. A literature search indicated that methodological support for team TNA has progressed but is still relatively modest.

Three team task analysis methods were identified: Hierarchical Task Analysis for Teams (HTA(T)) [6], Team Cognitive Task Analysis (TCTA) [7], and Team Task Analysis (TTA) [8]. HTA(T) and TCTA focus specifically on task analysis and are team extensions of the HTA and Critical Decision Method approaches respectively. TTA also focuses on the analysis of tasks and teamwork but goes onto consider the underpinning knowledge and skills required to conduct those tasks.

Two further methods were identified that could be used to identify gaps in current training provision and thus inform the development of requirements for changes in training provision to meet the gap. These were Task and Training Requirements Methodology (TTRAM) [9] and Mission Essential Competencies (MECs) [10]. TTRAM takes a task analysis as an input and uses methods for evaluating task difficulty and skill decay and the adequacy of existing training provision. Training gaps can then be identified and training interventions suggested. Generic guidance is provided on the training technology alternatives that could be considered, but specification of training environments is not addressed. The MECs approach uses a sequence of Subject Matter Expert (SME) workshops and surveys to develop sets of competency statements, lists of underpinning knowledge and skills, lists of associated experiences critical to the development of these competencies, and an evaluation of the adequacy of the current training provision for supporting competency acquisition.

Given the reliance of these methods on the experiences of the SMEs in extant training, these methods may be less well suited to analysis of an entirely new system with a new concept of operations such as the Queen Elizabeth Class carriers being acquired for the Royal Navy.

A report by Rugg-Gunn, Cunningham, Grimshaw, and Lawrence [11] was the only work found that outlined a Team TNA method. Their approach is split into two phases: analysis and design. They suggest that the starting point for analysis should be the determination of the key features of the team’s task in terms of team structure, scenario descriptions, communications links and factors that put the team under pressure, although only limited guidance is provided as to how this might be achieved. The use of organizational charts and scenario and timeline analysis diagrams are suggested. The use of HTA(T) is then recommended for the task analysis with an analysis of team communication and co-ordination requirements for the sub-tasks identified followed by an analysis of potential team errors. The analysis phase is completed with an evaluation of task training priorities using Difficulty, Importance and Frequency analysis. The design phase is split into three components: the establishment of training principles; the specification of training media and the measurement of team processes. The guidance on the first two components consists of a tabulation of the principles of team training followed by further tables identifying the match of types of training media to types of training task. Detailed guidance is provided on the development of measurement instruments. It does not directly address the specification of training solutions or consider the requirements for instructional staff and supporting systems.

In summary, no overarching approach was found which meets the requirement for a TNA method capable of addressing the complexity of identifying training requirements and specifying training solutions for team and collective training. The Team and Collective Training Needs Analysis (TCTNA) method described in this paper was developed for the MoD to address this methodological gap. The approach taken in developing the TCTNA method was to first develop a model of team training to underpin the analysis, then develop an analytical model and finally identify suitable methods and representational techniques to implement the analytical model. Following a short overview of the team training model that was developed and the structure of the TCTNA method, this paper focuses on the sequence of methods and representational techniques selected to implement the method, using a case study (the Local Area Surface Defence (LASD) of a warship against Fast Inshore Attack Craft (FIAC)).

The tctna method

In this section we discuss the Team Training Model and describe its derivation. Then we describe the architecture of the TCTNA method and discuss how it maps onto the Team Training Model.

A literature search was conducted to identify a team training model that that could be used to underpin the TCTNA method. No single model was found which provided a sufficient representation of the components of team performance, the interaction of the team with the environment and the functions of instruction and training delivery and the systems required to support these functions, all of which need to be addressed in TNA for a complete training solution to be specified. Therefore a new model of team training was developed as shown in Figure 1.

Whilst a detailed description of the formulation of the model is beyond the scope of this paper (for a detailed description the reader is referred to [12]), the key points identified in the literature that informed its construction merit discussion.

There are numerous models of team performance in the team performance literature, most of which are structured around inputs, processes and outputs and are often termed IPO models. Examples of such models include those of Hackman and Morris [13], Tannenbaum, Beard and Salas [14] and Weiner, Kanki and Helmreich [15]. Team processes are central to all such models and are thus represented centrally in the Team Training Model. Team processes are defined as encompassing both task focused activities, such as operating warship systems, as well as teamwork activities such as communication and task co-ordination, as explicitly identified in the NATO Command Team Effectiveness Model [16]. Common inputs identified in the IPO models include both team properties, such as team organization and cohesiveness, and individual factors such as knowledge skills and attitudes. Therefore, these elements are shown as inputs to team processes. The IPO models also commonly show team processes as having individual and team outcomes such as changes in individual motivation and team communications patterns. This is reflected in the Team Training Model by the feedback arrows from task outputs to team properties and team member characteristics. The final input to team processes shown in Figure 1 is the task environment which is shown as an input to team processes in the team training model with task outputs feeding back to the environment. This is based on Roby’s model [17] which takes an information processing approach and characterizes team activity as a response to cues from the task environment leading to actions that affect the environment. Consequential changes in the environment will present modified cues as inputs to the team.

For the purposes of TCTNA, the task environment refers to all the features of the real world that are of significance from the perspective of task performance. This can include the physical environment in which the team are located, the systems they operate, and other people and systems that they interact with. Arguably, it is the demands of the task environment that have the potential to put the team under stress. From observation of many US Navy exercises, Cannon-Bowers and Salas [18] identified a set of factors which could impact on team performance: threat, performance pressure, time pressure, high workload, high information load, requirement for team co-ordination, rapidly changing/evolving scenarios, incomplete/conflicting information, multiple information sources, adverse physical conditions, and auditory overload/interference. Klimoski and Jones [19] also identify resource availability as an influencing factor. We suggest that resource scarcity/depletion (such as running low on ammunition) is a potential stressor. With the ever-increasing number of IT systems with visual displays and facilities such as chat, visual overload may also be a stressor. We term this set of stressors environmental task demands and identify them as attributes of the environment that must be captured as part of TNA.

Team performance models do not generally reflect the development of team performance through the interaction of instructional functions, although the Tannenbaum et al model [14] does show team interventions acting on team processes. The representation of the interaction of instructional functions on the team training model is based upon the Team Training Cycle model put forward by Tannenbaum, Smith-Jentsch and Behson [20] which identifies the key instructional processes of pre-briefing practice sessions, conduct of practice, diagnosis of performance and the provision of feedback. This model has been used to underpin team training approaches such as Team Dimensional Training [21]. We have also added environment management functions into the model, as the instructional team will need to control events in the environment such as opposing force actions. Supporting systems for the instructional and environmental management functions also have to be identified. The dotted arrows in Figure 1 represent the monitoring of each of the aspects of team performance, the provision of feedback to team members, and the monitoring and control of the task environment. The architecture of the TCTNA method is shown in Figure 2.

The aim of the team and collective task analysis is to gain a detailed understanding of the nature of the task to be trained and develop collective training objectives. In short it needs to determine what needs to be achieved, by whom, in what environment using what resources, and then how the task is achieved. It is concerned not only with task analysis in the traditional sense, but also with developing an understanding of the context of the task in terms of the nature of the environment and its dynamics, the team structure and individual roles within the team. This aids understanding of the task and therefore the development of the task analysis itself and also informs the later stages of analysis. The purpose of constraints analysis is to identify factors that constrain the choice of training solution (such as cost, equipment availability and safety) as early as possible so that effort is not wasted in the evaluation of inappropriate solutions. It should be noted that “constraints” also includes “opportunities” that may exist to reuse existing instructional facilities or development opportunities. Training overlay analysis focuses on the instantiation of instructional and environmental management functions. This includes the identification of appropriate training methods and a range of instructional and other supporting roles that have to be filled to deliver these methods as well as supporting systems and resources required to support these roles.

Training environment analysis is concerned with the specification of training environment requirements in an instantiation agnostic form, in particular the fidelity requirements for each element in the environment. The final stage is training option evaluation in which training options are identified and compared both in terms of their ability to meet the specifications developed in the previous stages of analysis and other criteria that are identified such as cost and development time.

Training overlay analysis and training environment analysis follow on from the team and collective task analysis. However, they are quite tightly coupled and can be progressed together—each element in the environment requires specification in terms of both its fidelity and its overlay requirements such as how it is to be configured and controlled and the data to be captured from it. Training options analysis can only be undertaken once these specifications have been derived. Constraints analysis can be run in parallel with the team and collective task analysis and should be completed before training overlay analysis and training environment analysis are undertaken, since there may be constraints such as legal requirements that impact on methods and environments.

Application of the tctna method

This section demonstrates how the TCTNA method is applied to a typical collective training problem, focusing on the development of specifications for training environments. A case study is used to illustrate the method. The case study chosen is the use of close range weapons systems on board a warship by the Local Area Surface Defence Team to conduct Local Area Surface Defence (LASD) in open waters against a Fast Inshore Attack Craft (FIAC) threat.

Team and collective task analysis

The team and collective task analysis can be viewed as having three stages. The first stage, context analysis, focuses on what needs to be achieved, by whom, in what environment using what resources. Task analysis forms the second stage, addressing how the task is performed. The final stage is the derivation of training objectives. Data collection for the first two stages may be achieved by analysis of documents such as Concept of Operations (CONOPS) and doctrine, discussions with Subject Matter Experts (SMEs), and observation of the task (if the equipment or a simulation already exists).

Stage 1 context analysis

A useful starting point for context analysis is to capture the nature of typical scenarios for the task. Scenarios may be captured as a simple narrative text or in a more structured way in a table. Table 1 shows an example of a scenario description for the LASD task. It captures the effect that has to be delivered, the environmental conditions, the various elements involved (such as FIAC and neutral vessels), the initial conditions and typical events that might occur as the scenario unfolds.

At the same time as developing the scenario description it is useful to construct a context diagram which portrays the external entities that the team interacts with. An example is shown in Figure 3. The notation is derived from that used by Ward and Mellor [22] for describing systems contexts for real-time software design. The boxes indicate the external entities that the team interacts with and the arrows show the direction of the interactions. As there can be multiple types of interactions with each element, a supporting interaction table can be used to provide descriptions of the interaction sets, cross-referenced to numbers on the interaction arrows. Table 2 shows examples of interaction table entries.

Having established the external context, the next step is to characterise the internal context of the team which encompasses the team structure and the nature of its immediate environment, its interfaces to the outside world and its internal communications structure.

A first step in understanding the organisational structure is to either obtain or construct an organisational chart. An organisational chart for the LASD team is shown in Figure 4. The organisational chart is useful from a number of perspectives. Most obviously, it shows the composition of the team and can be used to identify sub teams. In this case it shows that LASD team is comprised of three sub-teams, the bridge team, the ops room team and the upper deck team. The bridge team comprises three warfare specialists (the CO, a Principle Warfare Officer (PWO), and the Force Protection Officer (FPO)), and three seamen (the Officer of the Watch (OOW) the Navigator and the Helmsman). The ops room team is made up of another PWO and a group of sensor operators. The Upper Deck team comprises port and starboard weapons teams. The other feature of significance is that it highlights that there are three different local environments, one for each team (the bridge, the ops room and the upper deck). Having identified all of the team members it is then possible to tabulate their roles and the expertise that they bring to the team as a consequence of their individual training.

The nature of each of the team local environments can be captured using further context diagrams and supporting interaction tables showing the equipment and interfaces that the teams use to interact with the outside world and each other. This is an important step in which attributes of the local environments which are likely to be of significance for training need to be identified. For example, each weapon position in the upper deck team batteries can be considered to be a unique environment, since each may be at a different height above water (and hence the targets), with a different field of view and so the operators will have different views of a scenario as it progresses. On the other hand, some of the General Purpose Machine Gun and Minigun positions are located within 10m of each other. This means that there is s significant requirement for coordination between the respective weapons operators in terms of target identification, prioritisation and allocation. Furthermore, each position may have different communications reception (personal role radios can be affected by ship superstructure in the line of sight). This has a command and control implication for the Bridge team.

It is critical to understand how teams communicate both internally between team members, between teams and with external entities. A communications matrix can provide a concise summary of the communications networks (supported if appropriate by a suitable network diagram). Table 3 shows a communications matrix for the LASD team.

The communications matrix is of particular significance, as it captures the communications systems that must be provided in the training environment. Instructional staff must also be able to monitor these communications channels. A similar matrix can also be developed to show who needs access to the other systems in the environment.

Stage 2 task analysis

We suggest that a suitable initial approach for conducting the task analysis is to apply the Hierarchical Task Analysis for Teams (HTA(T)) method [11], as the HTA method upon which it is based is both popular and enduring [23] and therefore likely to be familiar to the analyst. This extends traditional HTA notation by including the team members/teams involved in each task sub-component or goal. An example of the HTA(T) graphical notation is shown in Figure 5.

Each box shows the sub-goal to be achieved and the teams and/or team members involved in attempting to achieve the sub-goal. The goal statements in each box can be used to provide a hierarchy of training objective performance statements. Annett et al [11] also suggested a tabular format that captures key information about each element such as a description of the teamwork involved and assessment criteria. We have extended the tabular notation further to capture task inputs and products, critical errors and consequences and data capture required for assessment. The capture of inputs and task products is critical as the training environment must be constructed such that these inputs and outputs can be supported. The critical errors and consequences can be used to inform a risk analysis to determine training priorities. Data capture requirements for assessment of significance for the subsequent specifications of systems supporting instruction in the instructional overlay. Goals identified as having a significant cognitive component, such as goal 1.2 Evaluate Threats, could be explored further with a suitable cognitive task analysis method.

The final element of the task analysis is the identification of the environmental task demands. These can be presented in a table stating the significance of each type of demand (high, medium or low) with a description of what generates the demand. Table 4 shows some sample entries for the LASD task. These serve to reinforce the key attributes that the scenarios delivered in training must embrace. It may be that more than one table is appropriate in order to capture sets of related task demands such as tasks with scenarios that differ significantly from each other. For example, LASD in confined waters presents different challenges because of the land-based threat and may therefore present a different cocktail of environmental task demands.

Stage 3 training objective development

Having completed the task analysis, it is possible to conduct a risk-based assessment of which tasks need to be trained by evaluating the data collected on possible errors, their likelihood and the severity of the consequences. In the LASD case it was determined that all tasks needed to be trained. At this point training objectives can be derived. Performance statements can be taken from the HTA(T). In this case, high level performance statements would be: locate threats, evaluate threats and counter threats. The conditions for performance are defined by the totality of the context analysis. Standards for each aspect of performance can be extracted from the tables supporting the HTA(T). An example for “counter threats” would be “in accordance with defined doctrine”.

Constraints analysis

Before conducting further stages of analysis it is sensible to determine the constraints which will apply. The LASD task is typical of the many military tasks which require weapons effects to be included in training. The common major constraints are cost, safety and resource availability. Plainly, live rounds cannot be fired at manned targets—you wouldn’t get many volunteers to helm a fast attack craft being fired at by selection of GPMGs, mini-guns and 20 mm cannon! Similarly, there would be little appetite for firing machine guns and RPGs at one of Her Majesty’s operational warships. The cost of live rounds precludes extensive live firing, and range availability is also a factor. Consequently, simulation will be required for weapons effects. Costs of providing swarms of 20 or more FIACs are also high.

Training overlay analysis

The next consideration is whether the whole team needs to be trained at once or whether some sub-team training would be advantageous. Given that tactical decision making requires not only initial scenarios to be presented but also the efficacy of tactical decisions to be established by seeing their effects and providing the opportunity for subsequent revision of tactics as required, whole team training in a suitable environment would seem to be the most appropriate option. Classroom teaching of key aspects such as rules of engagement and tactics would also seem appropriate.

The main instructional roles for the practical exercises would be setting up, monitoring and controlling the environment in order to deliver appropriate scenarios which are reactive to team actions, and briefing, monitoring and debriefing team performance. Significantly, these would require the facility to track vessel movements, both of the ship and the many FIACs in a swarm attack, as well as weapons effects both from and against the ship. The facility to monitor communications channels would also be required. Sufficient numbers of instructors would be required to monitor all the individuals in each of the sub-teams. On this basis, it is possible to specify instructional overlay requirements for FIACs as shown in Table 5.

A significant feature of FIACs is that they are manned. As such their behaviour is determined by the person at the helm. Hence, in this case, a field has been added for knowledge and skills required by the helmsman.

In addition to the requirements to monitor and control individual elements, there is the over-arching requirement to be able to construct, deliver and manage scenarios which match the generic scenarios that have been identified and deliver the required environmental task demands. If the generic scenario specification provides sufficient detail it may be possible to use that as the overarching specification, otherwise a more detailed version could be developed at this point.

Training environment analysis

Arguably the whole of the analysis document can be treated as a specification for the training environments, since each section captures pertinent information. However, further information is required for each specific environment that is identified. Notably, fidelity requirements must be established for each element that is required in the environment. The necessary elements can be identified from the context diagrams. Further information about their required behaviour (dynamics) may also have been captured in the environmental task demands table.

A framework is required for defining the requirements for elements that are required in the training environment. Since the simulation option for training will frequently, if not always have to be considered, the approach taken for specifying requirements for environment elements is to consider them in terms of their fidelity requirements. There are numerous definitions and dimensions of fidelity mentioned in the literature. Orlansky cited in Darby [24] refers to the “semantic quagmire” that surrounds simulation terminology, of which fidelity is but one term. Liu, Macchiarella and Vincenzi [25] identify a number of commonly used dimensions of fidelity and also note their overlapping definitions. For simplicity, we characterise elements in the environment in terms of their physical and functional fidelity. Physical fidelity is taken as referring to the attributes of the element that we experience with our senses (look, sound, feel, smell, and temperature). Functional fidelity is taken as referring to how the element behaves. An illustrative sample of physical and functional fidelity requirements for the example of FIACs is shown in Table 6.

These fidelity requirements are based on the cues that would be required to for the evaluation of the threat, and would need to be delivered regardless of the implementation environment.

This approach to specification can be applied to all of the elements identified in each of the context diagrams. However, consideration also needs to be given to the physical environments in which the sub-teams operate. In the LASD case these are the ops room the bridge and the upper deck. A significant feature that has to be captured is the geographical separation of the three sub teams. For example, it is not possible for someone on the bridge to speak directly to the upper deck weapons teams or to someone in the ops room. Equally, as the gun crews in the weapons teams are dispersed around the ship, they cannot speak directly to each other. Therefore, having all of the teams in one room within in easy voice communication of each other would be an unrepresentative training environment from the communications perspective. It would also be unrepresentative from the perspective of performance monitoring as the bridge team would be able to monitor the actions of weapons team members whom they would not normally be able to see. Consequently, specifications must also be derived which capture the key features of each of these environments and how they are connected.

Training option evaluation

The first part of training option evaluation is the identification and description of appropriate training options. It is necessary to provide a separate overall description of each option which would outline the option and detail how the instructor roles would be instantiated and the overall architecture of the system.

In the case of LASD, given the constraints related to live firing and weapons effects, at least two training environments would be required: one to support live firing, and one to support full weapons effects from both FIACs to the ship and from the ship to the FIACs. It is probable that there will be a number of alternative technical solutions for the provision of each environment, particularly where simulation is an option.

Each competing option has to be evaluated against a set of criteria pertinent for the particular training problem that is to be solved. Typically, factors such as initial cost, development time, running costs and resource requirements would be considered. An evaluation of how each option meets the detailed specifications that have been developed in the training overlay analysis and the training environment analysis.

The specification table format can easily be extended to capture evaluation data by adding columns to the right, one for each option. In which the instantiation of the requirements is described. This could also be supplemented with check boxes using a traffic light coding system to provide an easy visual overview of the suitability of different options (red for non-compliant, amber for partially compliant, green for compliant) which can be backed up with written comments to support amber or red codings. A potential format is shown in Table 7.

Discussion

In this paper we have described a methodology for TNA for team and collective training that is underpinned by a model of team training derived from extant research, and is implemented using established human factors systems engineering methods and representations, supplemented with additional tabular representations where required. The TCTNA approach described yields detailed specifications for team and collective training solutions against which training options can be robustly evaluated.

The method has been applied by the authors, fellow HF researchers and military TNA specialists to a number of small scale studies and several large scale studies are ongoing. Our initial experience and feedback from colleagues suggests that the method provides a practical approach to the problem of conducting team and collective TNAs. The representational techniques have proved to serve both as useful templates to structure data collection and as effective tools for facilitating discussion with SMEs, both during data collection and during review of the outputs. That said, we can only claim face validity at the moment. Formal validation of the method is required and the outputs of ongoing studies will be formally evaluated.

Acknowledgements

This work was conducted under the auspices of the Human Factors Integration Defence Technology Centre, funded by the Ministry of Defence and sponsored by the Human Dimension and Medicals Sciences (HDMS) Domain of the Dstl Programme Office.

References

Authors

Dr John Huddlestone is a Senior Research Fellow in the Systems Engineering and Human Factors Department at Cranfield University, England. He is the Head of the Centre for Training Innovation Email j.huddlestone@cranfield.ac.uk, Tel +44 (0) 1234 750111

Jonathan Pike is a visiting Research Fellow in the Systems Engineering and Human Factors Department at Cranfield University. His research interests include Training Needs Analysis and the application of teamwork models in training.