Volume 15, Number 1, March 2012
Mutual Empowerment In Mobile Soldier Support
- * TNO, Kampweg 5, 3769 DE Soesterberg, the Netherlands.
Abstract
By Mutual Empowerment, strengths and weaknesses of humans and technology can be dynamically balanced, resulting in new concepts for mobile soldier support. One part of our research focused on developing the mutual empowerment mechanisms themselves—for example, for supporting soldiers with text messages that are relevant to their role, location or task; for increasing organizational awareness; and for predicting collateral damage. Another part of our research focused on well-founded methods for designing such systems in an iterative and coherent way. We have developed a modular user-centred design methodology which aims at developing a library of reusable components for mutual empowerment. We developed support tools for designing, prototyping and testing such components. This paper presents the methodology, the outcomes, and our reflections on the methodology.
Introduction
Military missions are increasingly carried out amongst the people. They are typically situated in dynamic and unpredictable environments, where soldiers are working in a mobile context, and where vast amounts of unstructured information are available. The soldier is faced with complex dilemmas and must weigh various concerns such as own safety, hearts and minds of the population, rules of engagement, and mission goals.
To support soldiers in their increasingly more complex task environment, a mobile support system could add significant value. Soldiers can be provided with historical, geographical or real-time information; with information of their own capability, their enemy’s capability, their team’s status, and so on. The source of this information can be sensors, own troops or third (civil) parties. Whereas mobile military support systems might seem like an endless opportunity, their successful introduction is far from trivial. This is due to problems such as overloading the user with information, inappropriate judgments of relevance, trustworthiness (by the system or the user), or user interfaces which are inadequate to the use context (such as not being able to use a touch screen device during an exchange of fire). Therefore, for successful design of such systems, a clear visionary goal and proper design and evaluation methodologies are essential.
In this paper we propose to face this challenge by developing mechanisms for Mutual Empowerment (ME). Our approach aims at making humans and machines part of a team, compensating each others weaknesses, and optimizing each others strengths. The underlying technology does not need to be flawless, but can improve overall team performance by being transparent to the user (providing information about its own functioning), by allowing the user to intervene in the system’s functioning (user empowerment), by connecting humans among themselves and by pro-actively supporting the user with potentially relevant information. One part of our research focused on developing innovative ME mechanisms. For example, we have investigated a system for supporting soldiers with text messages that are relevant to their role, location or task; a system for increasing organizational awareness; and a system for predicting collateral damage. Each of these systems adds novel functionality to the current generation of dismounted soldier systems such as DINA [4] which mainly offers blue-force tracking and simple messaging. Another part of our research focused on well-founded methods for designing ME systems. We have further developed the Situated Cognitive Engineering (SCE) method [8] to make it suitable for incremental modular Human Factors design. The method allows us to develop a library of reusable components for mutual empowerment—that is, functional modules (FMs). We also developed methodological tool support for describing, prototyping, and evaluating in a virtual reality environment (based on the serious gaming environment VBS2).
By incrementally developing Mutual Empowerment modules, it is possible to bring substantial innovations into the practice of mobile soldiers. ME allows to combine human and machine reasoning processes that complement each other (that is, artificial intelligence methods are not implemented as an autonomous expert system, but embedded in a human reasoning process). The modular approach allows testing and refining elements that are expected to have a major impact, and to implement the support elements step-by-step from a common vision. Developments in the consumer market can be included well in this approach (such as in social media and context-aware services). Coherence is established with a sound analysis of operational demands, a uniform specification of the general support concept, and a common ontology which underlies all FMs. We believe that this modular way of working is a valuable contribution to existing methods of user-centred design.
This paper presents our approach on modular development of Mutual Empowerment, tools for conducting research on them and the modules we have proposed so far.
In the next section, we discuss the principles of ME. After that, we discuss the methodology we employ to develop ME modules, followed by a description of the support tools we have developed for this methodology. In the subsequent section, we will describe four functional modules which are developed according to our methodology and the ME principles. We wrap up by discussing how the different FMs can be integrated into a working system, and present a conclusion afterwards.
Principles of mutual empowerment
Mutual Empowerment views humans and technology (computers, mobile support systems, robots, etc.) as members of equal value in a human-machine team [7]. Their different capabilities should be viewed as complementary: by imposing a task-division between human and machine which is optimal in the current situation, we can improve overall team performance. To characterize the implications of ME for system engineers, we defined three principles described in the following sections.
Human in the loop artificial intelligence
Automating an entire reasoning process often leads to a system which performs worse than a human would. This should not be a reason to abandon AI solutions in general, as many AI solutions can be effectively applied to parts of a problem [14]. In general, we can overcome this problem by following a long tradition in human machine interaction aiming to bring the human in the loop [16]. By applying the theory of levels of automation [10] to AI reasoning, we can divide the reasoning process in different parts, such that only the parts which can be done well by the machine are done by the machine, and the other parts are left to a human. Within a task, we can distinguish between suggest, decide and act tasks for which either the machine or the human can be made responsible [5]. For example, the machine can easily pre-select good courses of actions out of hundred possible actions based on heuristics, while next the human can decide on—and thus carry responsibility for—selecting the actual action out of the proposed ones, which the machine might then in turn help carry out. To apply this design principle, the human-machine interfaces must be well designed, and “black-box” AI reasoning must be avoided. Because the optimal task division depends on the situation, the division of work may change dynamically as will be further explained in the next design principle.
Mutual understanding and adaptation
Good teamwork requires good coordination which in turn requires a shared understanding of which tasks are performed by which team members and how they are performed. Based on this shared understanding, the team can dynamically adapt their task division and way of working to the current circumstances. In a human team, humans adapt and maintain common ground without paying explicit attention. In the case of human-agent teams, these properties require explicit attention in the development phase. For example, when a user knows the quality of the wireless network connection is bad, he or she can choose to communicate the message by using the radio instead of sending a text message. The same principle applies the other way around. When a computer knows its user is currently busy, the computer can choose to postpone a notification until a later moment [17].
Complementing existing work practice
We do not choose for a monolithic approach where a large umbrella system replaces the current ICT infrastructure and redefines all current work practices. This would require a top-down analysis of all key team processes, leading to an unmanageable project whose outcomes are too disrupting to implement. Our approach is bottom-up, where many functional modules are developed in such a way that the mutual empowerment, support and flexibility that they offer can be deployed alongside the existing automation. By itself, a functional module has a proven value, but in combination with other functional modules, the value could increase more than proportionally.
SCE methodology
The Situated Cognitive Engineering (SCE) methodology [8] is a specific form of concept development and experimentation (CD&E [8]), which is tailored to the human factors domain. Similar to other approaches for cognitive engineering [6],[9],[12], it consists of an iterative process of Functional design, Prototyping, and Testing. However, the explication and implementation of these aspects is different in our approach, as will be explained in further depth below.
Functional design
The goal of this phase is to develop reusable functional modules. The design specification of an FM consists of scenarios, ontologies, metrics, use-cases, claims, and requirements. These are illustrated in Figure 1 and are further explained below.
Scenarios are used to capture the domain background by describing a general comprehensive story. Use-cases occur within the larger scenario and are short and structured prototypical examples of the envisioned man-machine interaction. Requirements describe what the machine should be able to do in order to make the use cases possible. Claims described the expected advantages and disadvantages of the requirement. They are used to justify the requirements by describing the effect on a certain Metric that the requirement is expected to have. Ontologies provide a constrained vocabulary in which use cases, requirements and claims must be described. Examples of these concepts are given later in this paper in Figure 3, 4 and 6.
By organizing the functional design specification in this way, use cases and scenarios guarantee operational relevance; claims and metrics guarantee scientific soundness; ontologies guarantee coherence and reusability. To put it differently; requirements specify the what (design specification); use cases and claims specify the why (design rationale); scenarios, ontologies and metrics specify the for whom: the operational and domain-specific background against which the system specification is developed.
Prototyping
The goal of this phase is to implement the FM designed in the previous phase. This implementation captures the basic elements of the system, typically at a low technology readiness level (TRL). The different FMs are implemented separately, and loosely integrated in a single prototype. By loose integration, we mean that the different components can interact on a semantic level (using the shared ontology). Interaction on a technical level (such as shared network protocols) is not tested at this phase. Also, on the user interface level, the integration is still elementary. The FMs can be used side by side but have not been combined into a single coherent user interface. In the last section, we will return to this issue and discuss how they can be integrated into a single coherent system in a later phase of development (at a higher TRL).
Testing
The prototype is validated in an experiment or expert review session, and generates input for the next development cycle. More precisely, it is tested whether the requirements have the expected effects as was laid down in the claims. This may lead to a refinement of the requirements or the claims, or a decision to abandon the whole concept.
Typically, the more evolved a prototype is, the more extensive its evaluation is. In general, evaluation experiments can differ in fidelity and realism [15]. Fidelity indicates how close the test environment resembles the environment in which the mobile support tool is planned to be used. For example, we can perform low fidelity testing in a laboratory using a cognitive walkthrough, or high fidelity testing in the real environment using a real implemented system. Realism varies from one extreme—the real environment—to the other, a virtual environment. For example, instead of going to a real battlefield to test a prototype, the prototype can also be tested in a virtual environment with virtual characters.
We have tested our prototypes in a high fidelity virtual environment (Virtual Battle Space 2), which will be further discussed below.
Research toolkit
This section describes the tools we developed for supporting the research methodology described in the previous section. The research toolkit serves to create a fertile ground for developing innovative FMs for mobile soldier support.
The following subsections describe the research tools we developed for each of the phases in the SCE development cycle: SCET (Situated Cognitive Engineering Tool) for functional design; Trex for prototyping; and M4E for testing.
SCET
SCET is a web-based database tool which supports the specification of functional modules as illustrated in Figure 1. The tool serves several purposes: collaboration, reuse, and quality assurance. It improves collaboration between multi-disciplinary research groups by structuring research results in the same way and by adopting one and the same vocabulary (ontology). The shared structure of representation also improves re-use of earlier project results as requirements, claims and use cases can be imported from other projects which had a similar focus. Finally, the tool facilitates quality assurance, as “missing fields” can be automatically identified to confront the researcher.
SCET has been applied to formally describe all ME functional modules we have developed. To evaluate whether this tool actually serves the intended purpose (described at the beginning of this section), we conducted a questionnaire amongst the ME research team. The evaluation revealed that, after the researchers had familiarized themselves with the tool and methodology, they were better able to understand the work of others and to see the overall picture. In this way, the tool has had a positive contribution to enabling a modular approach for developing an ME system.
Trex
Because information sharing plays a central role in mobile soldier support, we have developed a dedicated prototyping tool called Trex, which focuses on ontology-based information storage and visualization. The purpose of Trex is to allow rapid prototyping of ME FMs and to try out different possibilities regarding what information is presented to the user, and how this is visualized. Trex consists of three components:
- Knowledge base: A central ontology repository in which information is stored at a central location. Besides storage, the repository also supports ontology reasoning. The ontologies are described in the web ontology language (OWL) [3], and stored using Sesame [13].
- Filter: The filter mechanism continuously extracts data from the knowledge base. The filter should be set in such a way that the right, and only the right information, is presented to the user. The filter is implemented using a dedicated query language for OWL.
- Projection: Projections form the User Interface element of Trex. A projection contains an input specification (specified using an ontology), and a graphical output screen. For example, we have a timeline projection which can process data points which are tagged using the Time Ontology, and show them to the user on a timeline. In a similar way, we have map projections, organization chart projections, and grid projections.
Trex is the underlying framework upon which most of the FMs described in the following section are built. Trex has served the purposes of ME development well. Different filters and projections can be easily selected, which allows the research team to experiment with different settings without too much implementation effort. Furthermore, the ontology-based setup of the tool matches well with the ontologies which have been developed in SCET in the previous phase. Typically, the ontology as developed in the previous phase is fleshed out further when implementing the FM in Trex. Finally, because OWL is a language which is both human- and machine-readable, we can use it to investigate human in the loop AI mechanisms—that is, the same information can be entered into the knowledge base by a machine or by a human.
M4E
After the prototype was built, we typically wanted to test the prototype in a simulated environment. To facilitate coupling the (Trex) prototype with a simulation environment, we developed the M4E (Man-Man-Machine-Machine Etiquette) environment [5]. In order to evaluate an FM, both human operators and the FM, need to be brought together into the same virtual environment. This requires the FM to gather information from and publish information to the virtual environment.
The M4E environment is designed to fulfil this goal in a composable and flexible manner, such that components (that is, an FM, an intelligent agent, or a simulator) can be reused, saving development costs. The M4E environment relies on the use of both commercial off the shelf (COTS) and proprietary simulation components that are coupled and integrated using simulation interoperability standards.
Currently, the M4E environment supports simulation components for human in the loop (Virtual Battlespace 2), constructive forces generation (VR-Forces), sensor suite simulation (TASTE), Beliefs-Desire-Intention (BDI) artificial intelligence (JADEX), several custom HMI interfaces, and Trex. Any, or a subset, of these components can be brought together to evaluate a human-machine collaboration concept.
We have coupled all FMs presented in this paper to the VBS environment using M4E. The M4E simulation environment has provided an effective way to perform high-fidelity, virtual tests on an FM.
Application to mobile soldier support
The introduction mentioned the potential benefit a mobile support system could give a mobile soldier team, assuming this system would be equipped with ‘machines’ (Functional Modules) able to function in a human-machine team. The listed specific capabilities of these FMs range from providing mobile soldiers with historical, geographical, to real-time information about for their own capability or teams status. To get insight in all the possible FMs and their potential use in an operational setting we followed the SCE methodology. Below we first introduce the generic scenario that was used to guide the development of the FMs by proving a comprehensive background story. Next we describe four developed functional modules. The last subsection describes the evaluation of the FMs.
Scenario
We developed a social patrol scenario as a background story for our FMs. We derived this scenario from an existing concept of operations called Trutta [11], and added recent military experiences derived from interviews with servicemen and women. For the scenario, a country ‘Trutta’ is placed in Central Africa, approximately where Uganda is. Within Trutta there is a conflict amongst the population: criminal groups are extorting the population and the Trutta government is unable to resolve the conflict. An international military coalition called Trutta Force (Trufor) performs a mission in Trutta. Within Trufor, Dutch troops work near the village of Mkonela in the south of Trutta. In our scenario, a Dutch Combined Arms Team (CAT) conducts a one-day social patrol into Mkonela. Their mission is to gain intelligence on the influence of a criminal group called the Klemasi People’s Front (KPF) on the population of Mkonela. This influence is exerted through enforcing specific crops on local farmers and taking their profits. The patrol starts at dawn at the basecamp, from which the CAT travels per Bushmaster towards an overwatch location near Mkonela. Here, the vehicles are parked in a circle shape and the soldiers exit the vehicles. From the overwatch position, the foot patrol starts towards, and through, Mkonela village.
The first half of the scenario is mostly non-kinetic with the CAT observing the village and cropfields, and having social talks with villagers. The latter half of the scenario evolves into a more kinetic scene with a Troops-in-Contact and a mass of people storming at the CAT resulting in fire support from the basecamp. The scenario spins around these two extremes to show the dynamics and complications in contemporary missions. Next to this, it illustrates the need for multiple sorts of mobile soldier support.
Functional modules
Below, four FMs we have developed using the methodology are described: Info-4-You, person profile, organisation awareness, and joint fires appreciation. Their relation to the scenario is described through use-cases.
Info-4-You module
The first FM that we encounter in this scenario is the so-called Info-4-You module. This FM taps in at the scenario when the CAT exits the bushmaster and commences their patrol in Mkonela. The CAT encounters several crop fields on either side of the road at the edge of Mkonela. Because specific crops are a possible indication of KPF’s influence, the CAT takes interest in the type of crops grown, the owner of these crops and how they acquired them. The question for the CAT is now: which crops are they looking for? Which information is already gathered about the specific crops they see? Should they find out about who owns them, or is this already known?
To retrieve an actual view on previously acquired knowledge and therefore on knowledge gaps the CAT can make use of the Info-4-You module. By means of this FM mobile soldiers can add unstructured information (free text) into a database. To this unstructured info the soldier can add structured info such as the importance and confidentiality of the message as well as when it expires. In addition, structured information is automatically added by the FM such as the time of the message, its location and the author. Although basic, this neatly demonstrates the Mutual Empowerment principle of letting humans and machines do those tasks they are good at. When a CAT-member requires certain information, it can execute a free text search and in addition filter the results on certain (structured) info. The FM actually searches all the available messages and retrieves and displays those that fit the search criteria and filters.
Upon encountering the crops, the CAT searches for ‘crops’ and limit the search results to messages posted within 300 meter of its current location. This way they can quickly retrieve what information is already known (for example, “the field north of this location belongs to farmer Kosinga”); and as a next step add unknown information (for example, “South of this location is a crop field with sugarcane of 50 by 50 metres”.
Following the SCE methodology, the situation above was first described as a Use Case by means of SCET. Subsequently, we developed and built a prototype of this FM. The last step, testing the FM was an expert review session (see the conclusion section). Figure 3 presents a second Use Case related to the Info4You module which specifies that information can also be automatically pushed to the CAT. This example clearly demonstrated the SCE ‘use case’ concept. Figure 4 shows an example of a SCET requirement including the corresponding claims.
Person profile module
Soldiers on social patrol must know about the people they encounter: who are they, what is their role, their history, what has been discussed with them and which agreements have been made? In order to win the hearts and minds of persons, it is important to recognize them and to be consistent in statements. The second FM, called the Person Profile Module, focuses on supporting this need.
We defined the following use case: when continuing their social patrol the CAT meets the villagers of Mkonela. Though they would like to talk to everyone, it is most important to find out who the owner is of the field of sugarcane they saw earlier. Using the face recognition functionality of the Person Profile module they can retrieve profiles of people known to them, see Figure 5. Next to basic information about a person such as a name, function/role and a picture, the profile provides entrance to so-called Conversational Reports. Whenever a conversation has taken place, a summary of this conversation is stored as unstructured free text with additional structured information; the participants, the importance and the general sentiment of the conversation. This way it only takes one glance at a personal profile to determine whether interactions are generally friendly or hostile. In our use case the CAT inspects several people from a distance and finally decides to talk to a farmer (instead of to, for example, a teacher) in order to retrieve information about the owner of the sugarcane field previously inspected.
| Use Case | Navigating in environment based on previous info |
|---|---|
| Description: | The CAT has as goal to execute a social patrol and is aided by the ME-tool (storing info from previous visits) in navigating its environment. |
| Goal: | A successful social patrol. |
| Actor: | Group Commander (GC), ME Tool (MET) |
| Pre-condition: | Mission goal retrieved: social patrol |
| Post-Conditions | - |
| Trigger | CAT patrol reaches a fork. |
| Main action sequence | GC: walks on road, reaches a fork. GC: decides to turn right. GC: after 100 meters the road ends and the GC decides to return to the fork. GC and MET: At fork GC constructs a factual information message reporting that the right road ends after 100 meter and sets as trigger a distance of less than 10 meter to this location. |
| Alternative action sequence | GC: walks on road, reaches a fork. MET: pushes a factual information message stating that the right road ends after 100 m. GC: observes info message and decides to turn left. |
| Requirements: | Factual Information Constructing and Saving Factual Information Push and Presentation VBS2-ContentRequirement-RoadFork VBS2-SesameInfoRequirement-Position |
| Requirement | Factual Information Constructing and Saving | |
|---|---|---|
| Type: | Functional | |
| Description: | The ME tool supports the construction and saving of factual information, including possible trigger conditions for pushing this information to people. | |
| Claim 1 | Because factual information messages can be constructed and saved, later on they can be searched, retrieved - then well pushed - and presented. | |
| + | Preseverance of factual information (what is where observed) Aids pushing of factual information to people it is relevant for (by setting trigger conditions) | |
| - | Too much involvement (attention) and time allocation on tool, discarding environment When wrong triggers set --> info does not reach right persons | |
| Use cases: | Investigating environment, in specific crops Navigating in environment based on previous info |
Organisation awareness module
Recall the CAT on a social patrol mission to Mkonela. Several things can happen that result in an adaptation of the task allocation in the team, or an adaptation of the mission plan. For example, they run into a person who seems to have interesting information, but they cannot communicate due to language problems. This raises the unforeseen need for a translator. The challenge is to optimally use the resources in the network to fulfill this need. There could be people with the translator capability in the own organization, or in an allied party, but also in NGOs that are active in the area, or other locals could translate. With all options in mind, a choice should be made based on other reasons, such as availability, authority and trust.
This example fits the trend that more military operations take place in a networked setting. For the military organization it means that operational teams vary more in composition, but also that there are more dependencies with external organizations such as coalition partners, NGOs or local government. The organization awareness module described in this paragraph aims at providing soldiers with more insight in the network of the operation.
Concepts of ME play an important role in networked perspectives on military missions (such as network enabled capabilities and comprehensive approach), where adaptive teams and resources exchange between cooperating parties should hold the added value. Such flexible way of cooperation requires understanding of the network: Who is in the network? Which organizations and which persons? What are their goals? What do they want to achieve? What are their needs? What are their capabilities?
With the organization awareness module we propose an information system that contains the existing knowledge about the organizations, missions and resources, and presents it in an intuitive way to the military users.
The information system relies on the organization ontology as illustrated in Figure 6.
The organization ontology specifies a knowledge structure with three main aspects: organization, mission and resources. Each of these is explained below. The organization contains all aspects of the organizational structure, such as roles with corresponding authorizations and responsibilities and hierarchical relations between roles. The mission contains all information related to the mission, such as the main mission goal, a division in sub-goals, and a mission plan of how to achieve those goals. The resources contain all information about the resources. These could be human actors or system resources with their capabilities and states (such as position, and availability).
In an operational organization, a tight interconnection exists between all the aspects above. For example, a human actor enacts an organizational role, and therewith takes up the responsibility for the mission objectives that were assigned to the role. In the organization ontology we specify the semantic relations between knowledge elements in such a way that these derivations can be automatically derived by the OWL reasoning system.
The interconnection between the different aspects also implies that changing one aspect has impact on the others. For example, if an actor is no longer capable to fulfill his or her task, the consequence could be that the mission plan is no longer executable, and needs a change.
We present the information in a user interface. Several cross sections of the information can be shown depending on the needs of a user. For example, a hierarchical tree is convenient to show the organisational structure in terms of superior relations between the roles. Another possibility is to combine the organisation awareness aspects with for example position information. Figure 7 shows such a geographical interface. Each soldier is plotted as a symbicon with their current task written below. The green and red bars on the sides indicate whether they are capable of, and authorized for, their tasks. In this particular shot, the one soldier is not able to communicate and is in need of a translator. By sending a request for the translator capability to this interface, the known resources with that capability will pop up.
Joint fires appreciation code
Traditionally, the use of weapon systems and ammunition is based on the effectiveness in terms of the probability of kill. However, modern and future conflicts require an increasing use of effects-based deployment. Such deployment aims more at influencing the thinking and acting of opponents, decision makers and the public than the traditional deployment. This requires a minimal number of casualties amongst non-combatants and minimal needs for reconstruction. These requirements are even more stressing for weapon systems and ammunitions with large areas of effect. Their deployment is therefore highly dependent on instant information from the operating area, such as the presence of civilians or the state and function of the buildings (like schools, places of worship, hospitals) to prevent collateral damage. Without such information, the decision to deploy fire support will have to be made on a higher military level, resulting in a significantly longer time to effect. In worst-case scenarios, the deployment will not take place at all.
When the CAT is near the end of their patrol, it is suddenly assaulted by insurgents. At this point, the commander must provide responsive fire support that protects and ensures freedom of manoeuvre to the patrol. The Joint Fires Appreciation Code will project lethal and non-lethal effects of weapon systems and ammunitions on a digital map of the operating area combined with up-to-date information from users in the area. This will provide the user a full situational awareness of the operating area, enabling the user to optimize the deployment to match the desired effect. Moreover, the decision to deploy fire support can be made on a lower military level as the effects of the weapon systems and ammunitions can be directly presented to the user.
Thus, the Joint Fires Appreciation Code will keep the human in the loop and will allow deploying fire support both more quickly and more efficiently than the traditional call for fire support. The Joint Fires Appreciation Code can be added to existing fire support tools without having to replace them; it complements existing work practice.
Evaluation of functional modules
As a first step towards a full-scale evaluation of the FMs in an operational setting, we evaluated the Info-4-You and Organisational Awareness module in an experimental setting. The experimental setting was constructed as follows. In the virtual environment VBS2, we created a scenario that would highlight the possible use of these FMs. Next, we used M4E to connect dynamic, real-time information from this virtual world to the FMs based on Trex. For this experiment the Info4You and Organizational Awareness (OA) module were integrated so questions could be send by means of the Info4You module to people with a specific task as specified in the OA module. Moreover, the OA’s geographical interface was extended to show the information messages left at specific locations.
The evaluation concentrated on the execution of a military mission in the virtual environment. Participants were two reservists with eight and five years of service that were on active duty in respectively four and two overseas deployments lasting 4 months on average. Firstly, the participants received detailed instructions on the FMs they could use during their mission. Next, they were briefed on their mission: a social patrol in the village Mkonela. They received detailed instructions on their route and questions they should answer. After they executed their mission they were requested to denote, for each function of the two modules, the prospective added value as well as its matureness.
The evaluation yielded practical desired (interface) adaptations but foremost insight in the potential benefit of the FMs for the military. The results have been documented and were used to validate the claims and iteratively refine the SCET requirements. The bottom-line of the evaluation is that the added value of the FMs for the mobile soldier is different than the added value for the military organisation. The mobile soldier in the field was judged by our experts to be most supported by the functionalities of the Info4You module that 1) actively push new information that is added to the system and linked to the soldier’s current location to him, and 2) allow him to directly ask questions to people with a certain role (nowadays this has to go through several organisational layers). The organisation as a whole was judged to benefit the most from the functionalities that 1) allow the pushing of context-aware questions to the mobile soldiers, and 2) support them in logging their conversations.
Integration of functional modules
This section describes some of the issues that will rise when multiple functional modules are combined and further developed at higher TRLs. The risk exists that they may end up hampering the soldier, instead of supporting him. For example, the last thing a soldier needs is a screen that lights up unexpectedly in the dark, giving away the soldier’s position, or a system that requires attention in the middle of an engagement with enemy troops. An effective information support system needs to account for the fact that information needs and processing capabilities are dynamic in nature. Furthermore, attention must be paid to which modality is used to convey a certain piece of information.
To gain insight in these issues, we have developed an information management system aimed at delivering the Right Message at the Right Moment in the Right Modality—(RM)3. In (RM)3, the various information streams that are available have to “compete” with each other, over a limited set of resources. These resources represent the information displaying and processing capabilities of the soldier system. Typical displaying resources include visual (screen), auditive (headset) and tactile (vibrating belt or vest) displays. The information streams in (RM)3 are grouped into categories, such as “enemy reports”, “route information” or “geo-spatial information”. For each category, (RM)3 uses the content and available meta-data to provide a relevance ranking. In order to do that, the situation and task of the soldier needs to be assessed. This information may be provided by the soldier directly, or may be derived from sensors, such as a GPS sensor, heart rate monitor or a sensor on the trigger of the soldier’s weapon. Typical modes of operation include “Troops in Contact”, “Surveillance”, and “Social Patrol”.
Besides determining the relevance of information, (RM)3 also “knows” how specific categories may best be displayed to the user. For instance, an enemy report can be provided as an update on a map, but may also be communicated by a vibrating belt. When possible, (RM)3 provides multiple displaying options, so that the system will still function if specific display modalities are (temporarily) unavailable.
Given a description of the task, situation, information categories and available resources, (RM)3 manages the information flow by determining if, when, and how certain types of information should be displayed. For instance, when the soldier is involved in a Troops-in-Contact, (RM)3 will deduct that the user’s cognitive resources are consumed already, thus limiting the ability to process additional complex information. (RM)3 will then withhold complex information until the cognitive resources are freed.
We believe (RM)3 is a promising information management system which we plan to adopt as an overarching system for combining the FMs described in this paper.
Conclusion
This paper presents our approach on modular development of Mutual Empowerment systems, and proposes four novel functional modules for mobile soldier support.
On a methodological level we have faced the challenge of modular and iterative development of mobile support systems while integrating domain and human factors knowledge. This requires the different FMs to be developed in a coherent way to allow future integration, and where possible, cross-fertilization.
Our solution consists of making the integration of domain- and human factors knowledge an integral part of our methodology. Furthermore, we improved coherence by developing support tools to establish a shared platform at the level of design specification, prototyping and testing.
During several expert review sessions, our four FMs were evaluated as novel, contributing to existing soldier support, and relevant to end users in the military domain. We believe that the operational relevance is in a large part due to the way we have engaged end users (as in participatory design [2]) and incorporated domain- and human factors knowledge in the design and development process. By standardizing the research processes, and by maintaining ontologies throughout all stages of development, we have seen cross fertilization taking place between some of the FMs. For example, we have separately developed the info4you module and the organization awareness module. At a later stage, we integrated them to enable message filtering based on a person’s role in the organization. Because our methodology supports modular development, this was a natural next step, because on a fundamental level (ontological and scenario-wise) the different FMs were aligned with each other.
We believe our approach complements existing approaches for human factors-based development by finding a balance between iterative prototyping in short cycles, and capturing the essence of an FM in early stages of design. This essence can be found in the design rationale which specifies why certain choices are made, and the ontological underpinning, which formalizes the domain and is a necessary condition for information sharing between different entities in a human machine team.
In the future, we will further develop the FMs, and do more extensive end user evaluations. Furthermore, we will apply our methodology and software tool suite in other domains as well.
Acknowledgements
This research is supported by TNO’s Defense, Safety and Security research project Mutual Empowerment and by the EU FP7 ICT Programme, in the Cognitive Systems & Robotics unit, Project #247870FP7 (NIFTi).
References
D.S. Alberts and R.E. Hayes, Code of Best Practice for Experimentation, CCRP Publication Series, 2002 (http://www.dodccrp.org/)
S. Bødker and S. Pekkola, “A Short Review to the Past and Present of Participatory Design”, Scandinavian Journal of Information Systems, Vol. 22, No. 1, pp. 45–48, 2010.
D.L. McGuinness and F. van Harmelen, OWL Web Ontology Language, http://www.w3.org/TR/owl-features/.
DINA-C2 Soldier System, http://www.teleplanglobe.com/index .php/products-and-solutions/c4is/dismounted-soldier-system.
W.A. van Doesburg, A. Heuvelink, P. Langeslag, and K-J. de Kraker, “Man-Man-Machine-Machine Etiquette: Towards an Environment to Investigate Behavioral Rules for Dynamic Man-Machine Teams”, Proceedings of the RTO Human Factors and Medicine Panel (HFM) Symposium on Human Modelling for Military Application, 2010.
E. Hollnagel, and D.D. Woods, “Cognitive Systems Engineering: New Wine in New Bottles”, International Journal of Man–Machine Studies, 18, pp. 583–600, 1983.
G. Klein, D.D. Woods, J.M. Bradshaw, R. Hoffman, and P. Feltovich, “Ten Challenges for making automation a ‘Team Player’ in Joint Human-agent Activity”, IEEE Intelligent Systems, Vol. 19, No. 6, pp. 91–95, 2004.
M.A. Neerincx and J. Lindenberg, Situated Cognitive Engineering for Complex Task Environments”, J.M.C. Schraagen (ed), Proceedings of the Seventh International Naturalistic Decision Making Conference (NDM7), Amsterdam, The Netherlands, 2005
D.A. Norman, “Cognitive Engineering”, D.A. Norman, and S.W. Draper (eds), User-Centered System Design: New Perspectives On Human–Computer Interaction, Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 31–62, 1986.
R. Parasuraman, T.B. Sheridan, and C.D. Wickens, “A Model for Types and Levels of Human Interaction with Automation”, IEEE Transactions on Systems, Man and Cybernetics, Part A, Vol. 30, No 3, pp. 286–297, 2000.
Pearson Peacekeeping Centre, Trufor Conops 90425 Operation Lancet Sword, LS 08 Tab B and Tab C to Appendix 1 to Annex C, NATO Unclassified, releasable to PfP EU, 090508, 2008.
J. Rasmussen, Information Processing and Human–machine Interaction: An Approach to Cognitive Engineering, Amsterdam, Elsevier. J. (1986).
Sesame. Openrdf.org, http://www.openrdf.org/index.jsp.
J.S. Sherwin, “A Computational Approach to Situational Awareness”, Journal of Battlefield Technology, Vol. 13, No. 1, 2010.
N.J.J.M. Smets, J.M. Bradshaw, J. van Diggelen, C.M. Jonker, M.A. de Neerincx, L.J.V. Rijk, P.A.M. Senster, M. ten Sierhuis, and J.O.A. Thije, “Method and Simulation Platform to Assess Human-agent Teams for Future Space Missions”, Special issue ‘AI Space Odyssey’, IEEE Intelligent Systems.
N.A. Stanton and M.S. Young, Vehicle automation and Driving Performance, Ergonomics, Vol. 41, No. 7, pp. 1014–1028, 1998.
J.W. Streefkerk, D.S.McCrickard, M.P. Van Esch-Bussemakers, and M.A. Neerincx, “Do (not) Disturb?! Using Context-aware Notification to Improve Effectiveness of a Mobile Patrol Task”, International Journal of Mobile Human-Computer Interaction, 2011, (accepted for publication).