Volume 6, Number 2, July 2003
Linking Individual Skills to Collective Outcomes: An Agent-Based Distillation Study
- 1 Land Operations Division, DSTO Edinburgh, PO Box 1500, Edinburgh, Australia, 5111.
Abstract
Training for the maintenance of collective or team skills is important to any organisation, particularly a military organisation. While it is generally accepted that a higher level of individual skill is advantageous to team outcomes, the details of how individual skills contribute is not well understood. A number of experiments have been carried out where the proficiencies of individual cellular automata agents are varied and collective measures of the effect are recorded. This gave an indication of the individual proficiency level required for a likely positive collective outcome. This paper focuses on investigating the effect on collective outcome of the degradation of individual combat skills, when applied to both uniform and mixed groups, using the agent-based distillation (ABD) Map Aware Non-uniform Automata (MANA), to simulate the behaviour of small teams. We used current (accepted) theory of how individual skills are lost over time to examine the training frequency required of individual skills to maintain an effective level. Several characteristics of individual skill degradation in small units have been predicted. This study has revealed the differences between the individual skills of shooting accuracy and stealth in their effect on small-unit outcomes. Terrain was shown to have little effect on unit performance for these scenarios. An important outcome was the observation of team skills decaying differently to individual skills, suggesting that a training schedule set to maintain team proficiency might differ from a training schedule set to maintain individual proficiency.
Introduction
Training for the maintenance of collective or team skills is important to any organisation. This is particularly true of a military organisation. While it is generally accepted that a higher level of individual skill is advantageous to team outcomes, the details of how individual skills contribute are not well understood [1-11]. This study attempts to gain some insight into this area using an agent-based distillation (ABD) to simulate the behaviour of small teams.
We have carried out a number of experiments where the proficiencies of individual cellular automata agents are varied and collective measures of the effect are recorded. This gave an indication of the individual proficiency level required for a likely positive collective outcome. We then examined the training frequency required of individual skills to maintain an effective level. Here we used current (accepted) theory of how individual skills are lost over time [1-9].
This work is part of a larger study to understand better the appropriate or effective mix of individual and team training and future work will address team-based skills such as communications and shared situational awareness.
MANA and abds
ABDs are a relatively new genre of simulation/analysis tool, which have been developed under the USMC’s “Project Albert” for non-linear analysis of Complex Adaptive Systems (CAS). ABDs in this context are low-resolution, abstract models that can be employed to represent a land combat environment [10].
MANA [11,12] (Map Aware Non-uniform Automata) is an ABD that has been developed by the New Zealand Defence Technology Agency (DTA). The MANA V1.0 User Manual gives an excellent description of the development and principles behind this ABD. The version of MANA used for the bulk of this work is MANA V2 and the Command Line Engine (both 1.9.13 build).
Generally speaking, MANA, like other ABDs in this area, allows the user to set up two or more teams (Red and Blue or neutral) of varying size and ability. The collective behaviours of teams are varied by changing the abilities (or personalities) of individual automata or entities and by manipulating some common data such as a shared picture of the situation. In this work we vary only the combat abilities of individual entities.
ABDs are relatively simple to set up and run when compared with traditional closed-loop iterative simulations. In as little as half an hour, a user can design a scenario, set up the entity personalities, and begin running the simulation on their desktop PC. It is also relatively simple, due to the speed at which MANA runs, to “data farm” the parameter space of interest. The flexibility in the ABD entity behaviours also allows investigation of some intangible aspects of combat [10]. As the entities do not operate under a set of hard-coded rules, but rather interact according to their personalities, a more realistic account of the non-linear nature of small-unit conflicts can be obtained. This study has utilised all of these advantages to farm the parameter space of the individual skills under investigation.
Experimental design
The effect of varying individual skill level on small units (9-30 entities) was investigated from two aspects. These were:
- individual skill variation within the combat unit (a uniform team); and
- degraded individual skills (below a base-level) of selected entities within the combat unit (a mixed team).
A series of experiments, addressing these two approaches, was run within a range of model scenarios in two phases: Phase 1a, characterised by holding the terrain constant across scenarios; and Phase 1b, where the entity behaviour type was held constant across the scenarios. The scenarios for each phase are summarised in Tables 1 and 2.
| Constant: Billiard Table Terrain Behaviour Level of Entities | ||||
|---|---|---|---|---|
| Entity Formations | Type 1 (Simple) | Type 2 (some realism) | Type 3 (most realistic) | |
| B Pln v R Sqd (30) (10) | S-1a-11 | S-1a-21 | S-1a-31 | |
| B Pln v R Sqd (30) (30) | S-1a-12 | S-1a-22 | S-1a-32 | |
| B Sn v R Sqd (9) (5) | S-1a-13 | S-1a-23 | S-1a-33 | |
| B Sn v R Sqd (9) (9) | S-1a-14 | S-1a-24 | S-1a-34 |
| Constant: Behaviour Level of Entities ( Type 3 - Most realistic from Table 1) Terrain Type | ||||
|---|---|---|---|---|
| Entity Formations | Type 1 (Billiard Table-BT) | Type 2 (Easy Going-E) | Type 3 (Close-C) | |
| B Pln v R Sqd (30) (10) | S-1b-11 | S-1b-21 | S-1b-31 | |
| B Pln v R Sqd (30) (30) | S-1b-12 | S-1b-22 | S-1b-32 | |
| B Sn v R Sqd (9) (5) | S-1b-13 | S-1b-23 | S-1b-33 | |
| B Sn v R Sqd (9) (9) | S-1b-14 | S-1b-24 | S-1b-34 |
In phase 1a, 12 scenarios were used, each with the same simple terrain type. This was a billiard-table (BT) terrain type, which represented a flat, no-feature, surface with perfect line-of-sight (LOS) in all directions and perfect movement in all directions for each entity. Type 1 represented simple entity behaviours where the two sides were forced into combat from opposite corners of the battlefield. Type 2 represented some realism, where the Blue entities moved in a simple patrol formation, and Red were encountered on sensors during the patrol. They then moved into combat following similar rules to Type 1. The final group of Type 3, representing most realistic, built on the Type-2 behaviours but had Blue pursue Red on contact. Red used shoot and run tactics. With these tactics in place, the Red entities are able to “get away” from Blue and at times are “forgotten” if they remain out of Blues sensor range for a sufficient period of time. The scenarios end when either a) a stop condition of all Blue or all Red eliminated is triggered, or b) 300 time steps is reached. This time step limit was determined as the length of time sufficient to allow combat to take place and survivors to continue to their assigned goal.
In phase 1b, the effect of terrain was examined. In the easygoing terrain type (E) scenarios variable movement rates were introduced (simulating open field with changing ground conditions). In the close terrain (C) scenarios we added LOS obstructions to simulate open forest with obstructions to both movement and vision.
Correlation between skills and personality variables
There are numerous variables and personality weightings within MANA that govern the behaviour of the entities. Of key interest during this study were the skills that were solely individual, that is did not depend on group interactions. To this end the variables governing the entity behaviours were assessed for linkages to real skills and whether these skills were individual or group-based. The individual combat skill variables used were: Single Shot Kill Probability (SSKP), Stealth (the probability of not being detected by an enemy) and Contact Drill Performance (on contact the entity became more or less stealthy for a defined period and then continued under its original parameters). Across all the experiments the Red team’s combat skills were kept constant at:
SSKP = 40%
Stealth = 40%
Contact Drill = None.
The Blue team’s combat skills were varied in the following way:
SSKP → 100 to 0%.
Stealth → 100 to 0%
Contact Drill → Delay 2-15 time-steps; stealth increased 0 to 100%.
The experimental design described in section 3 was applied to the three combat skill variables which resulted in 2 (uniform and mixed teams) × 24 (# scenarios) × 3 (# combat skills) = 144 individual experiments. The mixed team experiments were then broken down further to cover the different combinations. Due to the large number of experiments and associated data only highlights of the results are presented.
Measures of effectiveness
MANA is a two-sided (Red and Blue) conflict simulation, which collects attrition-based information on the two forces. The objective of the Blue side is to minimise its own casualties in an engagement whilst maximising Red casualties. The Measures of Effectiveness used were:
- numbers of Blue and Red casualties,
- the Loss Exchange Ratio (LER): , and
- Force Effectiveness Value (FEV): .
Results
Uniform team degradation
All of the following results are generic and fictitious and are not based on any current military doctrine, measure or performance results.
The trends and outcomes obtained from ABD’s, particularly turning points, are characteristics where change is noted. However, the absolute values of these trends cannot be translated directly to real scenarios as they are only indicative of behaviours that might be expected in real situations.
Table 3 summarises the key outcomes from the uniform-team study.
| Combat Skill Variable | Summary of Outcomes (from all scenarios and analysis) | Effect of Terrain |
|---|---|---|
| SSKP (accuracy) | SSKP’s ≥ 50% provided best outcomes for the units. Turning-point at SSKP ~ 20% (that is, non-linear FEV) | Terrain had no noticeable effect |
| Stealth (ability to use cover) | The more Stealth is used the more effective the outcome (there is no key feature) Linear degradation of team performance with Stealth. | The only effect from terrain was the FEV measure for Close terrain producing a higher overall trend. |
| Contact Drill | Effect on unit outcome was marginal. Some linear effect seen in “simple” scenarios. Linear degradation path dependent on the stealth factor. | The only effect from terrain was seen in the FEV measure for Close terrain producing a higher overall trend. |
Figure 1 shows an example of the SSKP characteristics for the FEV measure. (Note, the worse Blue’s outcome, the higher the FEV). This example shows the effect of reducing the SSKP on the FEV, presenting a near-linear trend until a turning-point at around SSKP=20%, where there is a significant deterioration in Blue’s performance.
Stealth varied against FEV is shown to be approximately linear in Figure 2.
Contact Drill was shown to have negligible impact on all measures for the most realistic scenarios. The summary of results for contact drill presented here is weighted towards the “most realistic” scenarios. The negligible effect reported is due to the scenario format with Blue unable to take advantage of the situation with Red using shoot and run tactics. However, in the “simple” scenarios (or a situation where the two sides are fixed in combat) it is found that the contact drill can have some advantages to the unit outcomes.
Mixed team degradation
The large number of experiments run for this section resulted in an extremely large data-set of outcomes for analysis. The key items of interest are summarised in Table 4.
| Combat Skill Variable | Summary of Outcomes (from all scenarios and analysis) | Effect of Terrain |
|---|---|---|
| SSKP (accuracy) | No more than 5 of 9 Blue entities can have their skills degraded from the rest and still eliminate Red and minimise own casualties. The less these entities skills are degraded the better the outcome. | Close terrain → lower overall casualties for Blue, LER and FEV (1:1 force ratio) and → lower overall casualties for Blue, LER and a higher FEV (3:1 force ratio) |
| Stealth (ability to use cover) | Linear decrease in performance as the number of entities with degraded skills is increased from 1→8 of 9, and increasing stealth makes only slight improvements to FEV. | Close terrain → lower overall Blue casualties, higher LER and FEV (3:1 force ratio). And → lower overall Blue casualties, LER and FEV (1:1 force ratio). |
| Contact Drill | Negligible change to unit outcome based on the number of entities degraded and there were no noticeable or significant changes for the level of degradation. | Only close terrain had a noticeable effect, producing a higher FEV. (1:1 & 3:1 force ratio) |
Other than those from the simple scenarios, the results could not be replicated or explained by simple linear estimates, implying some emergent or non linear behaviours are present.
Time for skill decay
In this section we used current knowledge of how individual skills are lost [1-9] to examine the effect of skill decay over time against team proficiency. SSKP results for uniform team skill decay and FEV measure were used. The decay function for the SSKP decay was the power function shown in Figure 4, derived from the literature [1-9]. The calibration of this curve in terms of time is arbitrary (not based on experiment) and meant only for illustration.
Using the power-function curve, the results from Figure 1 were transposed to replace the decay of SSKP with time in months, and so generate an illustrative plot of (FEV)-1 decay with time. The consequent loss in shot accuracy is shown in Figure 5.
In Figure 5 the turning point of the curve occurred much earlier (~2 months) than in Figure 4 (~5 months). This demonstrated that the team Measure of Effectiveness decayed differently to the individual skill, suggesting that a training schedule set to maintain team proficiency might differ from a training schedule set to maintain individual proficiency.
Conclusion
Several characteristics of individual skill degradation in small units have been predicted using an ABD. This study has highlighted that Measure-of-Effectiveness versus skill-level curves have a characteristic shape regardless of the conditions. Also revealed are the differences between the individual skills of accuracy and stealth in their effect on small-unit outcomes.
The team Measure-of-Effectiveness was demonstrated to decay differently to the individual skill; suggesting that a training schedule set to maintain team proficiency might differ from a training schedule set to maintain individual proficiency.
Although not exhaustive, this study has laid the groundwork for further investigation and, as one of the first studies in this area, has shown the utility of ABDs.
References
[1] Human Resources Research Organisation, “Retention of Combat Skills by Active Duty Personnel”, Report Number: AD-A034-127-1, 1957.
[2] T. Tremble, and C. Alderks, “Measures for Research on Small Unit Preparedness for Combat Effectiveness”, US Army Research Institute for the Behavioral and Social Sciences (ARI), Report ARI-RN-92-03, 1991.
[3] R. Wisher, R. Kern, M. Sabol and B. Farr, “Reacquisition of Skills by Combat Engineers Mobilised from the Individual Ready Reserve”, US Army Research Institute for the Behavioral and Social Sciences (ARI), ARI Research Report 1667, 1994.
[4] R. Wisher, R. Kern, M. Sabol and H. Sukenik, “Individual Ready Reserve (IRR) Call-Up: Skill Decay”, US Army Research Institute for the Behavioral and Social Sciences (ARI), ARI Research Report 1595, 1991.
[5] C. Stothard, and R. Nicholson, “Skill Acquisition and Retention in Training: DSTO Support to the Army Ammunition Study”, Australian Defence Science and Technology Organisation (DSTO), Report DSTO-CR-0218, 2001.
[6] T. Thompson, J. Morey and S. Smith, “Basic Rifle Marksmanship Skill Retention: Implications for Retention Research”, US Army Research Institute for the Behavioral and Social Sciences (ARI), ARI Research Report 1326, 1981.
[7] M. Sabol and R. Wisher, “Retention and Reacquisition of Military Skills”, Military Operations Research, Vol. 6, No. 1, p. 59, 2001.
[8] D. Bryant and H. Angel, “Retention and Fading of Military Skills:Literature Review”, Human Systems Inc for the Defence and Civil Institute of Environmental Medicine (DCIEM)—Toronto, DCIEM Report No. CR2000-070, 2000.
[9] R. Robinson, “Objective Measurement of Training Readiness”, US Army War College, Report ADA089418, 1980.
[10] Project Albert Web: http://www.projectalbert.org/, 2002.
[11] M. Lauren, M. Anderson and R. Stephen, “Exploring the Role of Troop Quality in Combat Effectiveness using MANA”, DTA Report 182, NR 1367, New Zealand Defence Technology Agency, ISSN 1175-6594, 2001.
[12] M.K. Lauren and R.T Stephen, “Map-Aware Non-uniform Automata (MANA) – A New Zealand Approach to Scenario Modelling”, Journal of Battlefield Technology, Vol. 5, No.1, 2002