A Computational Approach To Situational Awareness

I. Introduction

Situational awareness (SA) is widely recognized as a crucial element to human-in-the-loop operations [1,2]. In fact, SA is inherent to all levels of command and control (C2) operations, from the highest level decision-makers and policy-makers to the most detailed levels of action in an operational situation (OS). This paper focuses on a potential computational approach to a policy-maker’s SA, which can determine the course of strategic and tactical choices amidst a conflict.

Regardless of which level in C2, the accurate accomplishment of SA in an OS can be elusive. It becomes even more difficult when such a task is relegated to a calculating machine, such as a computer. For human implementation of SA, years of training and instruction contribute to better performance. For example, aircraft pilots get better SA with more experience [2,5,6]. But when it comes to a computational approach to SA, the results are somewhat frustrating. Two primary aspects to SA contribute to this difficulty: (1) a complete understanding of what mechanism is at play when humans integrate large swaths of data—that is, learning, remains lacking, and (2) the attainment of SA depends on future goals. Thus far, the human approach to SA severely outperforms the computational one.

Nevertheless, with an increasingly electronic battlefield [3], the necessity has emerged to perform SA using computers to aid humans. Furthermore, ‘despite some [now 50] years of research, AI has delivered very few successful contributions in this critical area’ [4]. In other words, a new approach could be useful for computationally implementing SA. Therefore, the following proposes a mathematical generalization from which a computational approach to SA can emerge. A demonstration of this approach will then follow along with its validation from real-world observation.

This introduction (Part I) has identified the current and future need for a successful computational approach to SA. Part II details the mathematical abstraction on which the subsequent parts of the paper are based. Part III describes the example case in terms of its mathematical generalization. Part IV then uses such an abstraction to describe a mechanism by which computational SA can be implemented on the example case. Part V validates the results of this application and conclusions are drawn in Part VI.

II. Mathematical generalization of SA

To pick an appropriate mathematical framework for SA, it is necessary to consider the inherent difficulty in doing so. In particular, the problem is that more data is not equivalent to more information [2]. Thus, a mechanism that accomplishes SA must provide a way to take raw data and translate it into useful information for the user. In the case of the policy maker, this means translating mounds of data into actionable information. But the way to do this is not so clear.

As a possibility, computer algorithms provide a way to take lots of data and translate it into smaller amounts. Furthermore, some computer algorithms can even filter such data to produce actionable information. For example, the PageRank calculation on which the Google search engine is based [7] employs a user-defined search criterion to rank useful information. In other words, Google transforms mounds of raw data related to the search criterion into actionable information from which the user can benefit.

But there are some added elements to SA that have thus far made it beyond the reach of most computer algorithms. In particular, an added element necessary to SA is an ‘advanced level of situation understanding and a projection of future system states in light of the operator’s pertinent goals’ [1]. In other words, if a computational SA is to be achieved then it must not only be able to integrate large amounts of data into a ‘situation understanding’ but it must also be able to predict ‘future system states.’ This is a prime feature of the Hierarchical Temporal Memory (HTM) algorithms discussed in Part IV. Additionally, ‘[as] such, SA presents a level of focus that goes beyond traditional information-processing approaches in attempting to explain human behaviour in operating complex systems’ [1]. In other words, a computational SA would have to reproduce the behaviour of humans ‘in operating complex systems.’ Considering the complexity of an operational situation (OS), this is surely an apt description of what a policy-maker is doing as he/she understands developing features of the OS to bring it to a satisfactory conclusion.

With the aforementioned aspects of SA in mind, it is possible now to establish a mathematical framework in which computational SA will be addressed. As the previous discussion on SA has alluded, an OS can be considered to be a complex system characterized by a set of phenomena. Labelling the complete set of its phenomena as $P$, the set of observable phenomena is then $V$, whereby it is understood that $V\subset P$ via Hayek’s observations on complex phenomena [11]. Furthermore, every observation ($x_i$) that makes up $V$can be considered to be a vector such that each component of $x_i$ is an observation on a particular feature of the complex system in question. Consequently, V can be considered as a vector space filled with n observations ($x_i,i\in \left\{1,\dots ,n\right\}$) in time.

With this mathematical abstraction in place, it is possible to recast the problem of SA as an exercise in creating manifolds ($M_j$) such that each $M_j,j\in \left\{1,\dots ,m\right\}$ represents a high-level feature of the complex system in question. Furthermore, the set of all high-level features thus comprises an understanding (U) of the system in question, that is$U=\left\{M_1,\dots ,M_m\right\}$. The U then is hypothesized to be synonymous with the SA on which one acts in the context of a predefined goal. In other words, high-level features of a complex situation constitute the actionable information in SA, not the minor details. Therefore, the aim of SA in this mathematical framework is necessarily to extract U from V.

A visual representation of this mathematical framework can be seen in Figure 1, which shows ten sets of observations on a given complex system exist from which an understanding of that system is generated—that is, n = 10 and m = 3.

Necessarily, there must be a mechanism in place by which the m manifolds are created, since this process would otherwise be arbitrary. So, the understanding (U) generated from these manifolds is thus dependent on the mechanism by which observations are grouped into perceived high-level features. In other words, the accuracy of U depends on the mechanism by which the m manifolds are created. A brief description of one such possible mechanism shall be reserved for Part IV in which the HTMs used to create a U are described.

III. Example case: Iraq 2003−2008

To illustrate the process by which SA can be accomplished within the proposed framework described above, an example case will be used: policy-maker SA amidst the Iraq conflict of 2003−2008. In terms of the mathematics above, all phenomena associated with this conflict comprise the set P_Iraq, where the subscript now and henceforth denotes the example case. In finding what the appropriate V is though, it is important to specify ab initio the desired goal for the future states of the Iraq conflict. In other words, the specification of V depends on the desired goal of those engineering the future state of the Iraq conflict. This is a crucial aspect to SA, as described above. Assuming the declared intent of the coalition forces is to establish stability in Iraq after the ousting of Saddam Hussein [8,9], any observations on P_Iraq that can contribute to the desired state of a stable Iraq will be crucial to include in V.

Obviously, since $V\subset P$, there are nearly innumerable ways in which a V_Iraq can be constructed, but the guiding aforementioned feature of SA is that only those observations that contribute to a desired future state are worthy of being included in V. Still though, there are measurable degrees by which any V can be constructed, since each engineer of a complex system has his/her own notions of what is important to understand a given $P$. For the purposes of illustrating this proposed approach to SA though, it will be assumed that the forthcoming V_Iraq constitutes a sufficient, though by no means complete, description of the Iraq conflict from 2003−2008. This is assumed because the proposed V_Iraq constitutes those metrics that are inextricably linked to a desired future state of stability in Iraq. The credibility of such an assumption is shown in Part V, when the computational SA is validated against human SA for this case.

Establishing a V_Iraq to use throughout this analysis, the Brookings Institution provides a panoramic view of Iraq from 2003−2008 in terms of observable metrics of stability [12]. Based on the fact that elements of security, politico-economic and surge tracking are crucial elements to stabilizing Iraq, it has been inferred that a suitable V_Iraq is extractable from this report. This report contains monthly data on these three crucial categories of phenomena observed in Iraq. The metrics shown in Table 1 have been chosen for their continuity over the time span in question and their ability to comprise a suitable V_Iraq for building a U_Iraq.

As is shown in the next section, the continuity in time of the evolution of each of these n = 16 metrics is crucial to the mechanism by which the manifolds comprising U are created. Although the V_Iraq proposed here is not a complete description of P_Iraq, it is hypothesized now that it is suitable for creating a useful U_Iraq. Necessarily, the validation of such a claim is tested in Part V, in which a qualitative expert U is compared to that generated in the next section.

IV. Mechanism for creating SA

This section describes the mechanism and shows the results by which a U is created from a V. In particular, the example case of how a U_Iraq is created from the proposed V_Iraq is shown. The mechanism described in this paper depends on Hierarchical Temporal Memories (HTMs) built on the Numenta Platform for Intelligent Computing [13]. An HTM is a set of algorithms whose construction is based on observations and conclusions from contemporary neuroscience in its investigations of the brain’s neocortex. The interested readers are referred to Hawkins [14] and George [15] for details on the concepts behind and the computational implementations of HTMs, but certain illustrating principles from these sources are explicitly mentioned here.

First, an HTM is a set of nodes that enact a series of algorithms from which a U of a given V is created. Each node in an HTM groups the data contained in V. When arranged in a hierarchy, all of the nodes form an HTM network in which the outputs of one node are passed along as the inputs to another, unless the node is accepting a part of V as its inputs. Although there are many aspects of HTMs that make them suitable to extracting a U from a V, the primary one to consider is that the temporal proximity of similar observations forms the basis by which a set of x_i’s are grouped into manifolds. This is why the temporal continuity of the metrics’ evolution in Table 1 necessary. Even though temporal proximity of similar observations forms the basis of the temporal pooling algorithms, these algorithms occur on top of the results of spatial pooling algorithms, which establish the degree of similarity between observations based on distance calculations. Although this explanation is painfully brief, the interested reader is referred to George [15] for details on these algorithms and the information flow. Suffice it to say though that HTMs can enact a rudimentary means by which raw data is translated into a set of high-level features embedded implicitly within that data. Furthermore, given the ties to neuroscience, HTMs provide a unique possibility both now and in the future for computationally accomplishing SA.

Although the above discussion on HTMs and their neuroscience background has been brief, it is hoped that the forthcoming analysis of the U_Iraq created from the proposed V_Iraq will demonstrate how a policy-making SA can be attained computationally with surprising accuracy. As a means of validation, the SA that has been attained quantitatively is then validated against a qualitative SA in the next section.

As opposed to a neural network, in which inputs are matched to preconceived outputs [16], the HTM creates the output space—that is, U, in the approach done here. In other words, there is no supervision of the process by which U_Iraq is formed and so what is termed an unsupervised network is employed on V_Iraq. For the sixty months covering May 2003 to April 2008, Table 2 shows the evolution of the U_Iraq created.

While the first two columns match the month with the value of $n\in \left\{0,\dots ,\text{59}\right\}$ during which an observation on P_Iraq is made, the third column specifies the most active temporal group at each time step. It is crucial to realize that the number of the most active temporal group output by the network is synonymous with the manifold number. In particular, m =10 manifolds constitute the $U_{\text{Iraq}}=\left\{M_0,M_1,\dots ,M_{\text{10}}\right\}$ created from this analysis. Therefore, eleven shades of stability have been recognized by the HTM in its analysis of the provided V_Iraq, where each of these shades of stability is the most probable high-level feature recognized at the indicated time points in Table 2. The question then is to see how this quantitative recognition of the stability status compares with the conclusions of those who are experts in policy-making SA. This is the focus of the next section.

V. Validation with qualitative SA

To validate the U_Iraq proposed in the previous section, it is necessary to compare this quantitative understanding with a qualitative one provided by experts who are presumably enacting it to secure the future stability of post-Hussein Iraq. As a source of such expertise, the quarterly reports from the United States Department of Defense, entitled ‘Measuring Stability and Security in Iraq,’ provide a qualitative understanding (U_Iraq) that can be compared with the one calculated above. The comparative analysis between the qualitative and quantitative understandings of V_Iraq is broken into five sections. These five sections correspond to the following time frames of the conflict:

• January 2004 to June 2005
• October 2005 to February 2006
• August 2006 to October 2006
• January 2007 to March 2007
• April 2007 to March 2008

In the course of the forthcoming comparison, it is revealed how the results of HTMs corroborate the assessments expressed by the SA of the policy makers.

Beginning with the first period, January 2004 to June 2005 (time points 8−25), the M₀ manifold is the most active temporal group. The question to ask though is whether this label corresponds to any particular state of stability witnessed in the V_Iraq. Referring to the quarterly report in which this time period is discussed, ‘the following [data] on overall attack trends indicate that the total number of incidents has been relatively level in the post-election period and remains below the election peaks’ [8]. Thus, the preliminary conclusion is that M₀ corresponds to a somewhat stable situation, but only examinations of future time periods will inductively verify this claim.

Continuing with the second period, October 2005 to February 2006 (time points 29−33), there is a noticeable period of change in the stability manifold the HTM recognizes in V_Iraq. During these months, the HTM recognizes new stability situations, which are labelled M₂, M₃, M₄, M₅, M₆, where each subscript corresponds to the most active temporal group number. Furthermore, just before this time period, M₁ replaces M₀ in August 2005 as the situation gets worse in Iraq. Comparing this quantitative account of the stability situation with the Defense Department’s qualitative one yields startling similarities [17]:

During this reporting period, the President of the United States, acting upon the recommendations of military commanders, authorized an adjustment to the U.S. force posture in Iraq, decreasing the number of combat brigades in Iraq from 17 to 15, a reduction of about 7,000 troops. [...] However, as expected during this period, the total numbers of attacks against Iraqi and Coalition targets have risen.

In other words, both the qualitative SA offered by the U.S. Defense Department during this time period and the qualitative one from the HTM notice a change in the stability situation. Furthermore, in light of the preliminary hypothesis that M₀ corresponds to a relatively stable situation, it seems that when any other manifold is recognized by the HTM the stability in Iraq during this time period begins to flag. The period of October 2005 to February 2006 justifies this claim because non-M₀ manifolds are most active during it. Although this does not prove the claim that the HTM is recognizing varying degrees of stability in V_Iraq, it certainly lends credence to it inductively. But, as with all inductive proofs, further comparative analysis is still needed.

Looking at Table 2, August 2006 (time point 39) constitutes somewhat of a transient fluctuation in the stability manifold. Based on the conclusions made thus far, this would indicate that there was degradation in Iraq’s stability during this month. The Defense Department says though, ‘[during August through October 2006] the total number of attacks increased 22%’ [18]. So the question then is why did the HTM not recognize time points 39−41 as being non-M₀ manifolds, i.e. instable situations? The answer lies in the fact that, since the Iraq OS is a complex system, the interdependency between all of its elements contributes to its observed phenomena, and to its stability. In particular, both electricity and water supply are key elements to P_Iraq. So any improvements in such elements as these will contribute positively to the stability of Iraq. In fact, ‘electricity output was ‘2% more than the previous quarter’ and new water projects have elevated the ‘supply of potable water by 35% since May 2006’ [18]. As Table 1 shows, electricity is explicitly contained in V_Iraq and so this information is directly affecting the U_Iraq created by the HTM. Furthermore, since the Iraq OS is a complex system, whose elements are inextricably dependent upon each other, the increased supply of drinking water also adds to Iraq’s stability during this period. Though this effect is only indirectly contained in V_Iraq, it is indirectly incorporated into the U_Iraq calculated here.

In fact, a more instructive comparison to the quoted qualitative U_Iraq comes from looking in more detail at the quantitative one. Instead of only looking at the most active group numbers of Table 2, Table 3 shows the details of the U_Iraq outputted during this time period.

• Table 3 shows a more in-depth account of the quantitative U_Iraq because not only are the top three candidates for most active manifold shown but also the belief values by which each manifold is recognized as such are shown below each label. As is known from Table 2, M₇ displaces M₀ as the most active group at the 39^th time point. But what is not clear from Table 2, is whether this transition to instability had noticeable precursors. Table 3 allows one to see that during time points 37−38 M₇ is the second most active group until it becomes the most active at 39. In other words, the HTM quantifies the coming instability in Iraq during June and July 2006. So not only is it true that non-M₀ manifolds correspond to decreased stability, but also HTMs quantify the possibility of instability in the future.
• Thus far, it inductively seems that the M₀ manifold corresponds to a stable OS situation in Iraq whilst all non-M₀ manifolds correspond to varying degrees of instability. Looking at the fourth time period, January to March 2007, it is hoped that further justification of this correlation will be found. Starting with the U.S. Defense Department’s reports, the qualitative U_Iraq offers a grim picture of Iraq during these months [21]:

The conflict in Iraq has changed from a pre-dominantly Sunni-led insurgency against foreign occupation to a struggle for the division of political and economic influence among sectarian groups and organized criminal activity. As described in the January 2007 National Intelligence Estimate, the term ‘civil war’ does not adequately capture the complexity of the conflict in Iraq, which includes extensive Shi’a-on-Shi’a violence, al-Qaida and Sunni insurgent attacks on Coalition forces, and widespread criminally motivated violence. Some elements of the situation in Iraq are properly descriptive of a ‘civil war’, including the hardening of ethno-sectarian identities and mobilization, the changing character of the violence, and population displacements.

In light of the conclusions made thus far, it is hypothesized that the HTM’s would recognize non-M₀ manifolds during this time period. Comparing the above account with the quantitative U_Iraq of Table 2, it is clear that in fact this is the case: M₈,M₉,M₁₀, are the most active manifolds during time points 44−46. Thus, the comparative analysis of this time period further justifies the claim that non-M₀ manifolds correspond to varying levels of instability.

The final comparison to validate the HTM’s U_Iraq against that of the U.S. Defense Department’s concerns the return to stability witnessed in Iraq from April 2007 to March 2008. If the previous inductions are accurate then U_Iraq should be fully extractable from the HTM’s output. Thus, in the forthcoming comparison, the HTM’s analysis will be offered first and then the qualitative U_Iraq will be shown to be in unmistakeable correspondence.

The HTM’s U_Iraq during this time period will now be discussed. If M₀ is the manifold corresponding to stability, Table 4 indicates that there is a potential for stability during the aforementioned period of near civil war. This is based on the result shown earlier that the HTM recognizes the potential for certain categories of stability via its second and third most active temporal groups. Since M₀ is the second most active group during February and March 2007, the HTM recognizes that there is a likely potential for future stability in Iraq.

At time point 47, M₀ becomes the most active group, thereby indicating that stability has returned to Iraq. However, given the small difference between the belief value of M₀ and either M₁ or M₂ during the 47^th−50^th time points, the stability is by no means secure.

Looking at August 2007 through April 2008, Table 2 shows that M₀ is the most active group for the remaining time steps from which U_Iraq was formed. Furthermore, although not shown here, the difference between the belief values of M₀ and that of the second most active group during this time period grow considerably. In other words, the HTM quantitatively recognizes that the situation is increasingly stable from August 2007 through March 2008.

• Now, this quantitative SA will be compared to the qualitative one offered by the U.S. Defense Department during the same time period. For ease of comparison, the three stability transitions noted above shall be labelled as follows: (1) the return to stability (February to March 2007, time points 45−46), (2) the provisional attainment of stability (April to July 2007, time points 47−50), and (3) the growing permanence of stability (August 2007 to March 2008, time points 51−59).
• Looking at the U.S. Defense Department’s U_Iraq, an account of the hypothesized return to stability (time points 45−46) reveals the following [19]:
• On January 10, 2007, the President announced the New Way Forward. The period covered in this report (February 2007 to May 2007) saw a greatly increased effort to secure turbulent areas to give Iraqis political space to implement reforms and pursue reconciliation among competing factions [1]. ... It is too soon to assess results. Positive indicators include a decrease in civilian murders and sectarian violence in Baghdad and in total attacks in Anbar Province, while negative indicators include the rise of high-profile attacks and expanded use of explosively formed projectiles.
• Thus, the claim that February to March 2007 indicates a potential for stability is justified with this statement. In other words, the HTM recognized M₀ as the second most active group only to have it fully takeover in the months afterwards. So a progression towards stability is seen during this period, but it could go either way, as the statement points out qualitatively and as the HTM acknowledges quantitatively via the belief values.
• Considering the hypothesized provisional attainment of stability, the HTM actually fills in some details about the efficacy of the New Way Forward. In particular, M₀ has moved up to become the most active group as of April 2007 (time point 47), though its position there is tenuous, as indicated by the small difference between its belief value and that of the second most active group’s during this time period. This is in direct correspondence with the above excerpt’s claim that ‘[it] is too soon to assess results.’

While the previous excerpt highlights improvements in terms of security, the Defense Department’s U_Iraq also describes improvements in the economic aspects of Iraq’s stability. The U.S. Defense Department says, ‘[while] significant challenges remain, several positive economic developments emerged’ [19, p. iv]. Even though surge tracking and security metrics have the most dominant presence in the V_Iraq used here, some economic metrics are explicitly represented (such as crude oil production and export, and nationwide unemployment rate), while many others are not. However, due to the interdependent nature of a complex system such as the Iraq OS, indirect effects on those observations that do constitute V_Iraq are necessarily incorporated into the HTM’s U_Iraq. Therefore, it is no surprise that both the Defense Department’s U_Iraq and that of the HTM agree on the increasing stability of Iraq with regards to this aspect as well as its security.

Considering the hypothesized growing permanence of stability, the Defense Department account of the situation is as follows [20]:

The security environment in Iraq continues to improve, supported by limited but important gains on the political, economic and diplomatic fronts. Violence levels have declined since the last report and Iraqi Security Forces (ISF) are gradually assuming responsibility for maintaining law and order and promoting stability. New strides have been taken in reconciliation at the national, provincial and local levels, and the Iraqi economy is growing. However, recent security gains remain fragile, and sustained progress over the long term will depend on Iraq’s ability to address a complex set of issues associated with key political and economic objectives. Violence levels are down throughout most of Iraq. Since the June 2007 report, deaths from ethno-sectarian violence are down nearly 90%. Total civilian deaths and Coalition deaths have each dropped by over 70%.

This excerpt covers January to March 2008 (time points 56−58)—that is, the latter part of the hypothesized growing permanence of stability. Nevertheless, during this time period, although not shown here, the belief value of M₀ pulls away from those of the other groups, indicating that the stability of the OS is increasing substantially. Thus, the HTM quantitatively recognizes a U_Iraq during this time period that is in correspondence to that offered qualitatively by the U.S. Defense Department.

In summary, the qualitative U_Iraq from such an expert source as the U.S. Defense Department is consistent with the quantitative U_Iraq offered by the HTM used in this analysis. In other words, it appears from the preceding results that the HTM provides a computational approach to SA on the level of a policy maker because it is able to recognize the difference between stable (M₀) and instable (non-M₀) situations. Furthermore, as policy makers must do by intuition, the SA offered by the HTM provides a means to quantify the tendency for alternate situations to occur. These alternate situations can either be desirable, such as increasing stability, or undesirable, such as degrading stability. But the utility of the HTM’s means of SA is that it may provide a way to quantitatively adjust policy in response to imminent, though not overt, threats.

VI. Conclusions

This paper has begun with recognizing the need for situational awareness (SA) even in the context of an increasingly electronic battlefield. In order to propose a computational approach to SA, a mathematical abstraction has been set forth whereby SA has been recast in terms of a manifold creation problem in a vector space. This vector space consists of observations on a given set of phenomena. A mechanism (Hierarchical Temporal Memories) has been proposed to enact this idea computationally. This mechanism has then been used to accomplish computational SA on the level of a policy-maker during the Iraq conflict after Saddam Hussein’s deposal. Finally, an expert source has been used to validate the claims made by the computational SA offered here.

Further research can now address whether such an idea can be useful in time critical decision cycles. In other words, the analysis offered here has been after the fact of the Iraq conflict. But this paper inductively lends support to the idea that such an approach could be used as an aid to policy-makers in the course of making the decisions that affect an operational situation (OS), such as the one analyzed here.

References

Author

Jason Sherwin is a PhD candidate at the Georgia Institute of Technology in the Department of Aerospace Engineering. He joined the Aerospace Systems Design Laboratory under Prof. Dimitri Mavris in 2005. He has a Bachelor of Arts in Physics from The University of Chicago (2005) and a Masters of Science in Aerospace Engineering from Georgia Institute of Technology (2006). He can be contacted by email at jason.sherwin@gatech.edu or by phone at +1 (404) 385-7485 .