Substantiating The Value Proposition For NCW—Metrics And Indicators

Understanding the problem

The concept of Australian NCW has crystallised since it was first embraced some five years ago, with a NCW Roadmap [1] developed to describe not only the aspirational NCW target states, but also to provide a timetable showing key projects in the Defence Capability Plan (DCP) [2] which are delivering capability towards achieving the desired NCW outcomes. Related projects are grouped in the NCW Roadmap and their relationships towards achieving specific NCW milestones noted. The NCW target states provide a high-level statement of the overarching qualitative properties sought over six key functional areas spanning force application, information superiority and support, command and control, force deployment, force protection, force generation and sustainment. The Australian NCW Concept is, however, an evolving one; the approach being based on “learn-by-doing”. Consequently it does not currently provide all details of what may be required to achieve the envisaged capability. Additionally, there is no definitive description of what constitutes a satisfactory outcome, no quantitative measures and no progress indicators articulated for judging achievement of progress towards some satisfactory outcome.

The NCW Roadmap does, however, outline a model-test-model approach, using the military exercise environment, to gain insights as a basis for further developing NCW concepts and capabilities. The questions arise ‘How can we tell what progress has actually been accomplished towards achieving the NCW outcomes sought?’ and ‘How will we know when the targets have been met?’

In one sense, the NCW Roadmap already provides a high-level means of monitoring progress though the simple checklist of milestones and the timetable for achieving them. However, there are no explicit performance criteria to establish whether each milestone has been achieved or not; the only indicator being the completion or otherwise of identified projects. Also, while systems engineering specifications may be available for individual capabilities delivered by projects with associated measurable objectives, there is no overarching specification that could be used as the basis for which to measure and assess the performance of the aggregate Australian NCW capability implemented. Determining how each of these projects might specifically contribute towards achieving the overall target states described in the NCW Roadmap poses a significant challenge. Furthermore, technical, schedule, cost and capability risks may arise from a number of sources, inhibiting the full potential of the envisaged Networked Force [1]. To develop an alternate approach to a simple checklist, a deeper understanding of the characteristics, attributes and measures associated with the aspirational NCW outcomes is needed.

How does NCW capability manifest itself

NCW capability in a warfighting environment has been described as an organising principle, having its genesis in the dynamic growth and competition principle that has emerged from the Information Age Economy [5]. Adoption of net-centric concepts has ushered in a new era in warfare through a fundamental shift from attrition style warfare based on the functions of platforms and the massing of force, to a distributed, decentralised, networked approach using modern information and communications technologies (ICT). This potentially faster warfighting style, also proffered to be potentially more effective, is based on massing of effects of smaller, distributed force elements and is characterised by new concepts of speed of command and self-synchronisation. These concepts have been formalised as fundamental tenets of Information Age Warfare [6].

The Australian concept of NCW proffers five basic premises based to a large extent on these fundamental tenets; but also incorporates the aspects of professional mastery and mission command central to Australian warfighting doctrine. This may then be used to focus the problem of NCW evaluation to one of collecting scientific evidence relating to these premises as manifested in the exercise and operating environments. In order to monitor progress, it then becomes a matter of agreeing to specific criteria to measure and establish a baseline NCW capability rating and establish agreed progress criteria for future ratings to provide a suitable basis for ongoing comparison and assessment.

Establishing a basis for measurement

While the notion of NCW is based on networking sensors, engagement systems, and decision makers, Moon differentiates the net-centric approach from traditional networking through human interaction only arguing that the advent of computer networks, widespread digital communications, and the supporting software and processes, enables the NCW behaviour sought [7]. At its core, NCW is thus achieved through using an information network, built on ICT infrastructure, to connect sensors, command and control (C2), and engagement systems, thereby supplanting and augmenting many of the functions previously undertaken by humans. But he acknowledges that insertion of technology alone will not realise the intended benefits of NCW; consideration of the human dimension being an important factor.

Moon suggests that, while the concept of networking sensors, C2 and engagement systems is not new, dating back to at least the British Integrated Air Defence System of the Second World War, the introduction of computer networks, along with supporting information technology, and their integration with widespread digital communications is at the heart of the much-heralded advantage derived from NCW.

It may therefore be useful to think of NCW-related capability as being generated by an assemblage of a number of component capabilities, each of which has certain characteristics such that, when brought together, the collective behaviour is net-centric. The components can be associated with the respective NCW grids and the underlying networks in terms of infrastructure and their use across all elements of the fundamental inputs to capability (FIC). Operating together they might be expected to manifest the net-centric behaviour being sought by the Australian NCW concept. Net-centric component capabilities are thus considered at the heart of, and enabling, NCW.

What should we measure?

This provides a basis to ask some questions of the participating elements in a Force and to explore emergent net-centric behaviour. Specifically, the following questions may be asked:

• What information do force elements have that is of value to share?
• What is their ability to share valued information?
• Given the ability to share valued information, to what extent is this information shared?
• How effectively is this shared information used?, and
• What effect did sharing this information have on the outcome?

These questions resonate with a dissertation by Gartska on desirable qualities and metrics for NCW [9] where he articulates the concept of a value chain based on the original tenets of Information Age Warfare as shown in Figure 1. These are expressed in the context of four domains: physical, information, cognitive and social.

Gartska introduces a set of related quality factors which are encapsulated in the Net-Centric Operations Conceptual Framework (NCOCF) [10]. However, this framework is information-centric, and does not take into account quality factors relating to the effector capability, particularly changes relating to strike and protect capability, which one might expect to be an integral consideration to net-centric warfighting capability. Similarly, it does not incorporate consideration of force sustainment target states, such as those described in the Australian NCW Roadmap, which might also be expected to have significant influence on mission outcomes and overall warfighting capability.

A value proposition adapted from the NCOCF is therefore proffered to examine what are the significant contributors to NCW outcomes in an Australian context as described in Figure 2. This then gives us a basis for measuring net-centric properties in the context of a Networked Force, where such a Force has been instantiated specifically for an operational deployment. The net-centricity of the Force can be measured in terms of the collective net-centric behaviour demonstrated during the measurement period in the context of the NCW value proposition, where the assembled Force comprises components with varying net-centric potential and combat power.

This gives rise to a set of qualities or evaluand derived from the value proposition as described in Figure 3. A single Figure of Merit (FoM) can then be determined from the evaluand to reflect the aggregated value of the net-centric behaviour measured.

The extent of net-centricity can be examined in four domains as described in the NCOCF:

• Physical Domain—where the actions of strike, protect, and manoeuvre take place, and where the physical infrastructure resides;
• Information Domain—where information is created, manipulated, value-added and shared;
• Cognitive Domain—where perceptions, awareness, understanding, decisions, beliefs and values of individual participants are considered; and
• Social Domain—where force entities interact, share information, share awareness and understandings, and make collaborative decisions.

Introducing the net-centric evaluation framework

An essential aspect of the evaluation activity is to capture a description of the Networked Force that is being evaluated. The description of the Networked Force, its characterisation, the value proposition, and its ensuing metrics and indicators are encapsulated in the Net-centric Evaluation Framework (NEF) shown in Figure 4 This provides the overarching guidance and analytical constructs for evaluating progress in implementation of NCW capability.

In the context of the NEF the Networked Force comprises all force elements in the instantiated Force for a particular exercise or deployment, notwithstanding their individual differing abilities to connect via a network. The net-centric component capabilities of the Networked Force can be described in terms of their technology, processes, organisation, and personnel facets thus including a broad range of systems aspects contributing to their net-centric capability.

The first set of measurements will characterise the baseline NCW-related performance and force effectiveness. Subsequent measurements of the same net-centric attributes, within an appropriate context, will provide indications as to what changes have occurred and whether any improvement in warfighting capability and force effectiveness have been realised with respect to the NCW value proposition. Changes in net-centric component capability can be determined by examining the changes from one measurement period to the next in the technology, processes, organisation, and personnel capabilities. Changes in the net-centric component capabilities and their measured behaviour can be compared to see if there is evidence supporting a causal relationship between the changes in net-centric component capabilities, changes in net-centric behaviour, and changes in force effectiveness. This will allow examination for evidence to link changes observed to the relevant changes instigated across all facets (for example, FIC) through the respective Service strategic planning initiatives and delivery of NCW capability through the respective acquisition projects. These changes can then be evaluated against various criteria to ascertain the extent to which progress has been achieved over some specified time interval, and whether this progress is acceptable.

Central to the ability to relate measurements from one period to the next, is ensuring the appropriate context is provided so that meaningful comparison can be made. Context dependencies include, but are not limited to:

• the threat environment—the effectiveness of the Force will be directly impacted by the level of threat levied against that Force, and hence the level of risk the Force is exposed to;
• the operating environment—the physical environment will have profound impact on the Force, including which force elements are deployed, what role they are assigned to fill, constraints on movement, modes of communication available, and means of engaging the adversary; and
• roles/tasks—the performance of Force elements can differ when undertaking different functions.

Accounting for the network dimension

Integral to the analysis is the ability to identify participating nodes and links to assess the extent of connectivity supported in the physical and information domains for the different information types and flows. The systems and equipments of the Networked Force are identified in the Force Composition construct within the NEF and may be described in terms of their sensor, C2, information networking and engagement functionality. Examination of these will identify sources and sinks of both organic and non-organic information, facilitate evaluation of the quality of both types of information, and determine the Force’s ability to share this information in the physical and information domains. It is also useful to describe the nodes and links in terms of the information systems and services which support the creation, dissemination and use of the information, as shown in Figure 5.

Accounting for the human dimension

For the Social Domain, participating organisations, teams, and individual personnel, their relationships and interactions are all identified within the NEF Force composition construct. Training, competencies, and experience of personnel, teams and organisations can thus be examined in terms of their potential contribution to the NCW value proposition. Additionally, specific staff positions can be examined to determine whether they have improved their situational awareness through improved quality of information and/or access to information, and the degree to which understanding has been achieved and shared. The extent and quality of collaboration can be examined along with degree of planning synchronisation achieved. Command structures and doctrine can also be examined to determine whether C2 agility has been achieved and whether this has been translated to Force agility. Doctrine can also be tested to determine whether a Force is fully exploiting the potential of being networked. Finally, the degree of synchronisation in resulting actions can be ascertained, along with the extent to which the commander’s intent was achieved, and whether there was an overall successful mission outcome.

Establishing a basis for data collection

While some attributes may be measured in absolute terms, for others it is often more useful to consider them on a relative scale with respect to “sufficiency for use”. Two classes of attributes may then described: “objective” measures, which are situation independent; and “fitness for purpose” measures, which are situation dependent.

Evaluations can be obtained by eliciting ratings from subject matter experts (SME) with a warfighter perspective, e.g. from military personnel participating in the deployment or military exercise. While individual ratings may diverge because of their subjective nature, at least the extent of agreement obtained can be quantified, and any significant divergence in viewpoint or contentiousness noted during reporting.

Evaluations can be made using a scoring method, for example a 5-point rating scale describing levels of attainment in net-centric characteristics towards achieving the desired outcome. Table 1 gives such a scoring method where ‘1’ indicates a decrement or no improvement, and ‘5’ indicates very significant achievement or improvement.

Since no attempt has been made to ascribe an absolute level for each attribute, ratings will be dependent on context. However, it is anticipated that insight may be gained into what levels of capability are sought and where improvements might be better effected. This is similar in principle to the construction and use of Capability Maturity Models (CMM) as developed by the Software Engineering Institute, Carnegie-Mellon University [11].

Attributes and metrics

The value proposition can be expressed as a set of higher-order evaluand, E(k). It then remains to expand on a set of lower-tier metrics M(j,k) associated with each evaluand, E(k), that may be used to characterise the Networked Force. A list of nominal lower-tier metrics, M(j,k) associated with the higher-order evaluand E(k) is provided in Table 2.

Determining a figure of merit

A value for each metric M(j,k) associated with each evaluand E(k) can be determined by computing the arithmetic mean of the individual samples or scores relating to the respective metrics. This is described by the equation as follows:

where x(i,j,k) is the value of the i^th score for the respective metric M(j,k), and N(j,k) is the total number of measurements made for each metric M(j,k).

Computing the standard deviation of M(j,k) will give an indication of the variability of the score for each metric, which may flag that there are factors or underlying causes which may be impacting on the particular metric under scrutiny (e.g. differences in infrastructure).

A value E(k) for each evaluand can be determined by computing the arithmetic mean of the metrics M(j,k) associated with that evaluand, which is described by the equation:

where M(j,k) is the value of the j^th metric associated with evaluand E(k), and N(k) is the number of metrics to be included in each evaluand computation. The MOE can also be derived in the same manner from a number of lower-order metrics where:

where MOE(p) is the measured value of the respective MOE metric, and P is the total number of MOE metrics.

The respective metrics M(j,k) are not necessarily independent. Consequently the inter-relationships between the respective metrics M(j,k), the associated evaluand E(k) yielding the measure of net-centricity, and the Degree of Force Effectiveness (MOFE) may be non-linear, indeterminate, or difficult to represent mathematically. However, the primary interest here is not to derive an absolute quantitative measure, but to provide an indicator of net-centricity, with sufficient sensitivity and discrimination to be able to measure and track changes over successive measurement intervals.

To avoid placing arbitrary emphasis on any particular metric, no attempt has been made to include weightings. Weightings could be applied to different combinations of metrics and evaluand to reflect different priorities or emphases for reporting purposes to different Communities of Interest (COI); however, the main emphasis should be on providing consistency in the actual computational algorithm used over time during different measurement intervals.

An overall Figure of Merit (FoM) or indicator of net-centricity relating to the value proposition can be determined by computing the area within the graph plot of the respective evaluand E(k) for the measurement period as notionally represented in Figure 6; a change in the area within the respective ‘web’ diagrams will give a broad indication of the extent of overall change in net-centricity detected over successive measurement periods.

This aggregate FoM can also be compared with the measured Force Effectiveness or MOFE over successive measurement periods to explore the relationship between the FoM and the MOFE, and explore whether the new NCW capability being delivered is achieving the intended capability improvement. Monitoring the change in the shape of the web diagrams over time will also give insight as to whether some evaluand have been impacted more than others by capability delivered since the last measurement period, and to explore whether these changes can be attributed to particular capability implementation initiatives.

Stakeholder reporting

Individual metrics and evaluand may be collated or combined in various ways for the respective Communities of Interest, providing different views depending on the context in which they are being considered and the purpose for which they are being used. An example of an NCW ‘reporting profile’ is shown in Figure 6. Throughout, an aggregated rating on a scale from 1 to 5 has been sought for each evaluand of interest. Progress can then be reported from each stakeholder’s perspective in terms of the relative extent of the change measured.

This approach recognises that many NCW qualities sought are subjective and thus attempts to evaluate them in relative rather than absolute terms. The method should thus be tolerant to changing aspirations or emphases, as a new FoM determination will still reveal how satisfactory or otherwise the measured performance is compared to the current expression of the capability goals sought. Data gathered on the net-centricity aspects of a Force under different circumstances should be comparable, provided the appropriate operational context is taken into account.

The individual evaluand and FoM derived thus provide an indicator of the aggregate change in net-centricity determined over successive evaluation intervals. Noting the changes in delivered NCW capability through acquisition projects at specific epochs, provides a basis for examining whether the investment is actually achieving the desired NCW capability outcomes and whether satisfactory progress is being made towards the stated goals. Such indicators can also highlight issues relating to people, technology, process and organisation considerations, which may impact on the desired outcomes. It is envisaged that this flexible approach to measuring and reporting net-centricity will allow new nuances of cause and effect to be explored as the NCW concept evolves.

Conclusion

A value proposition based on the Australian premises of NCW, has been articulated then used to evaluate emergent net-centric properties of a Networked Force in a known warfighting environment. This provides a basis for examining progress being achieved in implementation of NCW capability. A set of higher-order evaluand and lower-tier metrics have been developed based on the value proposition; a Figure of Merit was then developed to represent the aggregate net-centric behaviour measured. The attendant method for data collection and analysis has been outlined for examining this value proposition through the evaluation of net-centricity in the context of a Networked Force instantiated for a specific military exercise or particular operating environment. The NEF provides the analytical constructs to support the ongoing evaluation of NCW capability being delivered through acquisition projects within the context of the NCW Roadmap. The NEF as outlined in this paper was successfully applied during a recent major military exercise in support of assessing the NCW capability of a Networked Maritime Task Group.

References

Authors

acknowledgements

This paper was prepared under the auspices of the Network Centric Warfare Science and Technology Initiative, with the support of Command, Control, Communications and Intelligence Division, and Land Operations Division, DSTO. The author would like to thank Dr. Terry Moon for his inspiration and guidance, and the thought provoking comments from Dr Mark Unewisse during preparation of this paper. The author also wishes to acknowledge the significant assistance of Dr Peter Chatelain of DSTO, and Mr. Mike Roberts and Mr Bob Lewis from Jacobs Australia for their valuable contributions in the data analysis and reporting activity for the application of the methodology in the military exercise environment.

Meredith Hue received her Bachelors Degree in Electrical/ Electronics Engineering with Honours in 1979 and a Masters Degree in Business Administration in 1993 from the University of Adelaide. Meredith has a broad background in management, systems engineering, and systems architecting, in both industry and Defence, in the areas of real-time systems, combat systems, and military communications. She joined DSTO in 2001. Her research interests are in the areas of Systems of Systems methodologies, mobile networks, architecting methodologies, and modelling and analysis of C4ISR architectures and technologies underpinning future warfighting concepts. Telephone: +61 8 8259 7290, Facsimile: +61 8 8259 4193, Email: meredith.hue@dsto.defence.gov.au