Volume 2, Number 1, March 1999
Message Text Formats - A Solution to the Problem of Interoperability
Abstract
Effective Command and Control Systems depend upon timely and consistent information exchange between headquarters and operational units. Information can be sent and received by a wide range of systems and in an ideal world everyone would have the same system, the same database, the same operating system, the same computing platform, and so on - thus making information exchange much easier to achieve. However, that is not the case and it would be only the most optimistic of person who would acknowledge that this could ever be achieved, particularly in allied and combined operations. This short paper introduces the concept of interoperability using message text formats (MTF) as defined by the NATO standard ADatP-3 (Allied Data Publications number 3), Australian ADFORM (Australian Defence Formatted Message) or US MTF (United States Message Text Formats). In order to do this it is necessary to explain what is meant by interoperability in terms of unambiguous information exchange. As an example, from time to time, use is made of the Systematic COTS product IRIS. IRIS has been accepted by NATO as a product capable of preparing and validating ADatP-3 messages and in the USA has been approved for inclusion in the DII-COE (Defence Information Infrastructure – Common Operating Environment) as well as being approved for US MTF 97, 98 and 99. By using modern software products like IRIS, the problem of interoperability can be resolved using formatted messages.
Introduction
Effective Command and Control Systems depend upon timely and consistent information exchange between headquarters and operational units. Information can be sent and received by a wide range of systems and in an ideal world everyone would have the same system, the same database, the same operating system, the same computing platform, and so on - thus making information exchange much easier to achieve. However, that is not the case and it would be only the most optimistic among us who would acknowledge that this could ever be achieved, particularly in allied and combined operations. This short paper introduces the concept of interoperability using message text formats (MTF) as defined by the NATO standard ADatP-3 (Allied Data Publications number 3), Australian ADFORM (Australian Defence Formatted Message) or US MTF (United States Message Text Formats).
How do we define interoperability?
NATO, in AAP6 (Allied Administration Publication number 6), has the following definition of interoperability: “The ability of systems, units or forces to provide services to, and accept services from, other systems, units or forces and to use the services so exchanged to operate effectively together”.
NATO’s Management document MC245 states: “There is a NATO requirement that automated data systems, whether NATO or nationally owned, used by the forces of NATO, be interoperable; the extent of the interoperability between specific systems is to be determined and agreed according to the information exchange requirements of co-operating forces.”
For the purposes of this article, interoperability is defined at three levels. The first is the connectivity level. Communication systems can be connected together; for example the telephone rings and someone will pick up the receiver. Next there is the information exchange level; people can talk to each other in the same language. Solving interoperability problems at these first two levels is normally fairly straightforward. The problem arises at the third level, at what is called the ambiguity level. Do we understand each other? The following is an example of the problems that can occur because of the ambiguity of language. A pilot is flying a mission looking for enemy activity when he receives a voice message in his headphones “Bandits at One O’clock”. What does that mean? Well it could be one of two things. Either it is time-related or it could be position-related and if he gets it wrong, it could be fatal. Fortunately, in this case, the pilot knows what is meant. However, in other circumstances, he may not. We cannot afford to be misunderstood in a critical situation. So let us slightly refine the NATO statements on interoperability, so that INTEROPERABILITY can be summarised as being:
“… the UNAMBIGUOUS exchange of INFORMATION and SERVICES, between automated data systems to enable CO-OPERATING forces to OPERATE EFFECTIVELY together.”
How do we achieve this unambiguous exchange of information? One answer is to use Message Text Formats (MTF).
What are message text formats (MTF)?
Message Text Formats have been designed specifically for Military Information Exchange such as official orders, reports and procedures, for example, SITUATION REPORTS, ENEMY SITUATION REPORTS, PERSONNEL REPORTS, AIR TASKING ORDERS and many others. In the NATO ADatP-3 catalogue there are over 200 different message types, all of which have been designed to meet specific operational needs. The power of the MTF lies in the fact that it is structured to a well-defined set of rules and every message type is constructed from the same basic set of rules.
This section describes the basic ADatP-3 rules. Variations in other formats such as ADFORM and US MTF are described later.
Messages consist of one or more SETS including one MSGID SET which identifies the message type. SETS start with a SET Identifier and end with a double slash (//). Some SETS can be repeated, either individually or as part of a group known as a SEGMENT. A FIELD within a SET starts with a single slash (/) and contains a number of characters. For example:
SETNAME/FIELD1/FIELD2/FIELD3//
The basic structure of an ADatP-3 MTF is that the FIRST/SECOND SETS are EXER/OPER. These two SETS are mutually exclusive and are only used when messages should contain Exercise Names or Operational Code words. The THIRD SET is MSGID. This is the Message Identifier SET; it is MANDATORY and must appear in all NATO ADatP-3 messages. Most ADatP-3 messages also contain a FOURTH SET, which is REF. This is an OPERATIONALLY DETERMINED SET used to identify appropriate references and is usually repeatable.
Following the FOURTH SET there are any number of SETS dependent on the Message Type. The SETS can be interspersed with FREE TEXT SETS. There are three single FIELD FREE TEXT SETS in ADatP-3. AMPLIFICATION (AMPN) which is used to provide additional information only on the preceding SET. NARRATIVE (NARR) which is used to provide additional information on a preceding GROUP of SETS. REMARKS (RMKS) which is used to provide additional information on the whole message. These SETS are not defined as part of the Message Format. There is also a special FREE TEXT SET called a GENTEXT which has an additional FIELD used to identify the TEXT subject. The GENTEXT SET is defined as part of the Message Format.
SETS and FIELDS are designated as being MANDATORY (M), CONDITIONAL (C), or OPERATIONALLY DETERMINED (O); they can also be repeated (*).
An example of an MTF Structure is given in Table 1.
In the table, “Seg” identifies the start and end of a SEGMENT. A SEGMENT is a group of SETS, which must occur together. SEGMENTS can be repeated and nested – that is, a SEGMENT within a SEGMENT. “Rpt” indicates that a SET can be repeated and “Occ” identifies the SET status or occurrence. “SET ID” is the SET Name.
FIELDS can contain LEGAL VALUES. These are values that have been predetermined and one of these values must be used every time the FIELD is used. Some FIELDS have ALTERNATIVES. Examples of these are FIELDS that contain positional or date-time-group information. Again the alternatives have been predetermined.
It is not possible to describe all of these rules in a short article, suffice to say it is the very nature of the structure and the rules that enable MTF to be a suitable vehicle for interoperability.
Who owns MTF?
The ADatP-3 MTF catalogue is developed and managed by the NATO Interoperability Board (IOB) based in NATO HQ in Brussels and there are similar organisations in Australia (ADFORM Section, Doctrine Development Wing, ADFWC) and in the USA, DISA-JIO (Defence Information Systems Agency-Joint Interoperability Organisation). These organisations produce for their respective systems the “Baseline”, which is the suite of message types that has been agreed for OPERATIONAL USE. Currently NATO is producing Baseline 10 and Baseline 11 is expected in the Autumn of 1999. Although the IOB is custodian of the standard, the USERS and the SPONSORS are the Allied Defence Community. All NATO members have representatives at the MTF Working Group and all members can have a say in what message types there should be as well as the structure and purpose of those message types. It is worth mentioning at this point the role of Systematic Software Engineering in this process. A number of years ago the Systematic product IRIS was chosen by the German MoD, on behalf of NATO, under Project 6 of the Nunn initiative, to be the NATO ADatP-3 Automated Message Handler. Consequently, Systematic has an obligation to ensure that IRIS is maintained and developed in line with the latest changes and baselines being produced by IOB. To ensure that this happens there is an IRIS Configuration Control Group (ICCG) Meeting, which meets three times per year and is attended by Systematic, NATO IOB and representatives of countries using IRIS. Thus the product is able to deliver functionality consistent with the needs of the latest version of the ADatP-3 standard.
Other MTF formats
Although the ADatP-3 and ADFORM standards, originally, have been derived from US MTF, there are now a number of differences between the standards and it is worth while
looking at some of them in more detail. The three major differences to be examined are the use of Characters, Conditionality Notation and the definition of Fields.
The legal Character set in ADatP-3 are Alpha Uppercase Only (A), Numerical 0-9 (N), Specials such as full stop, comma, hyphen, brackets and question mark (S) and Blank denoted as space (B). The latest version of US MTF extends this to include Lower Case (L) and an Extended set of specials (E). ADFORM uses the standard ANSB but also adds Octal Numbers (C), Quantified Numbers such as kilo for 1,000 and Mega for 1,000,000 (K) and specific use of special characters in Free Text (X), (D) and (T).
The way the standards handle the use of CONDITIONAL or OPERATIONALLY DETERMINED FIELDS and SETS are typified in Figure 1, in which the use of ADFORM Conditional Notation (ACN) is compared with the Structured Notation from ADatP-3. The diagrams are screen dumps taken from the IRIS Definition System (IRIS/DEF) and show a message type called MISREP. In both Standards the first SET in a message can be either EXER or OPER. Note the way that the two standards describe this occurrence and the differences between the short succinct Structured Notation of ADatP-3, (2) P ([1] @), from the more comprehensive statements for ACN (WHEN OPER IS PRESENT: EXER IS ABSENT;). Note also the two different ways in which the standards handle the requirement to ensure that the MESSAGE ID field (FIELD 1 in SET 3) contains the name of the message type to which it refers (in this case MISREP). Another difference between the standards which, can be seen from the diagram is the way that ADFORM uses ACN to define the use of the FREE TEXT SETS “AMPN” and “NARR”. In ADatP-3 these FREE TEXT SETS are always left to the discretion of the person compiling the message.
The methods by which Fields and Field contents are defined in the standards again are different. ADatP-3 uses the concept of Field Format Index Reference Numbers (FFIRNs), Field Use Designators (FUDs) and Data Items whereas ADFORM uses only Data Elements and Data Items. An example of this difference is the “LOCATION” field. In ADatP-3 the field is given a FFIRN (four-digit number up to 9999) and the way in which the field is to be used is contained in the FUD (three-digit number); that is, LAT/LONG, UTM, Place Name or Bearing and Distance, and so on. ADFORM does not use the concept of FFIRNs or FUDs but uses Data Elements (nine-digit number) in place of both. US MTF uses the concept of FFIRNs and FUDs but the numerical definition used is not aligned with ADatP-3.
The differences between the standards identified above exacerbate the problems of achieving International Interoperability but high light the need for systems capable of handling all of the standards and all of the versions of the standards. IRIS is one of the software packages available that is able to do just that.
Other messaging system standards
The Message Handling portion of a message is defined by standards such as ACP 127, JANAP 128, DD173, ACP126m, STANAG 4406. These protocols are used to define details such as message precedence, classification, address details, date time group, routing indicators, and other attributes required for the message to be sent by the communications system and are commonly known as the “Envelope”. The contents of the message, or the Message Body, can either be Free Text or Formatted Text where the latter is provided by one of the standards under discussion in this paper. An analogy frequently used is that the body of a message can be likened to a person writing a letter, putting the letter in the envelope and then handing it to the Post Office to be sent. The letter is the MTF, the envelope is provided by the Message Handling protocols and the Post Office is provided by the communications system. Examples of an ADatP-3-like message with ACP127 envelope are shown in Figure 2.
Summary
Considerable time and effort is spent by the USA, Australian Defence Force and NATO to ensure the consistency and availability of the Messaging Text Format Standards. Armed Forces are able to exchange information with each other providing they are using message types from the same version of the Baseline. Message Text Formats have been designed to be readable by both MAN and the COMPUTER, ensuring interoperability during automated and manual processing.
How are MTF used in automated data systems?
Because the message types have been formatted to a controlled set of rules, a message prepared using MTF rules will only contain information essential to the military function. This is different from an e-mail message, which is a free text message and can contain everything or nothing with no particular order or structure (thus making it difficult to extract any information of real value). However, the formatted message can be parsed automatically and the important information extracted for decision making. Using the power of the computer and modern software tools it is possible to extract that information and put it where it can be viewed most effectively. As illustrated in Figure 3, this might be into an operational database, as an overlay to a map or into a spreadsheet along with data from other sources or with other messages for comparison and action.
In the reverse situation it is also possible to generate automatically new messages from third party applications. All of these actions can take place fully automatically, semi-automatically or manually. Messages do not have to be generated and received by systems having the same architecture, database, operating system or computing platform. Because the MTF structure is oblivious to the Command and Control or Communications systems being used, true interoperability can be achieved. As illustrated in Figure 4, a FIELD containing information or data generated in one system can be read by another without having to know from what type of system it was generated.
In the past, a number of Command and Control Systems have used MTF and hard-coded the message types needed for the system. This has worked well, but the information exchange is frozen at the time the system was defined and developed. The process of adding new message types and updating old ones as new MTF Baselines are produced, is time consuming and costly. In addition, interoperability between co-operating nations is only possible if the countries involved are using the same Command and Control System or their respective Command and Control Systems contain the same suite of message types from the same Baseline from the same MTF standard.
IRIS
These problems of keeping systems modernised in terms of Information Exchange have been overcome by using Commercial-off-the-shelf (COTS) software tools such as the Systematic product IRIS. The IRIS suite of products have been designed specifically with interoperability in mind; they have also been designed to ensure flexibility in development and operation and thus operational Command and Control or Communications systems containing the IRIS product can be easily updated when the ADatP-3 standard changes. The software has three main product areas: management of the formatted message standard using the IRIS Definition System (IRIS DEF); operational usage of the chosen formatted message types with the IRIS Message Formatting System (MFS); and development and integration of messages using IRIS Application Libraries as well as a new tool called the Information Modelling Tool. The following sections give a brief overview of each area and show how they interrelate and thus provide a powerful method of using MTF to achieve interoperability.
IRIS def
IRIS DEF is used by a central organisation to develop and manage Message Text Formats on behalf of a country and its Armed Forces. The latest Baseline is obtained from NATO IOB on CD-ROM and downloaded into IRIS DEF. Here it is possible to add new message types for internal usage only (that is, a country’s own specific message types), and to add help text to assist operational users on how to use the message type in their system and in their own language if necessary. The output of IRIS DEF is a MISSION FILE. The MISSION FILE can contain the whole of the NATO Baseline with the rules for all of the message types or a sub-set of it; that is, land messages, maritime messages or air messages depending on the function of the system. It is also possible to include message types from other standards such as US MTF, OTHT GOLD and Australian ADFORM as well as different versions of those standards. For example NATO Baseline 10 messages can be mixed with US MTF 97, 98 or 99 and at the same time contain message types for a country’s own internal use and in its own language.
Currently, IRIS DEF and its associated Management Module is being used by NATO IOB and the US DISA JIEO to manage and produce the ADatP-3 and US MTF standards respectively.
IRIS MFS
Having decided which message types are needed for a particular system the MISSION FILE is produced and is downloaded into the operational system that contains IRIS MFS. IRIS MFS knows nothing about messages until such time as it receives all of the rules from the MISSION FILE. However, when an operator selects a particular message type from the MISSION FILE, IRIS MFS automatically and dynamically configures the screens necessary to prepare and validate the message. IRIS MFS has the tools necessary to prepare and validate messages automatically, semi-automatically or manually as required. It also has the tools for managing those messages within the system and handing those message onto a communication system for onward transmission. IRIS MFS handles the protocols for a number of different communications systems from ACP 127, JANAP 128, DD173, ACP126m, and e-mail and there is an IRIS General Exchange module to provide standard interfaces to other systems. IRIS MFS has also been integrated with MS Exchange, LOTUS NOTES and a number of different X.400 systems. Incoming messages can be distributed to a nominated role (military function) and processed manually, automatically or semi-automatically. The messages can be read in their original form or parsed and the message contents transferred to a third party application such as a database, GIS or spreadsheet.
Application libraries and information modelling tool
For the systems integrator developing Command and Control or Communications systems there are a number of tools available to enable both the IRIS software and the message contents to be integrated into the end-user system. The IRIS Application Libraries are a suite of IRIS servers with a declared set of Application Programmers Interfaces (API) that can be used by the integrator to best suit his or his customer’s needs. In addition, there is the Information Modelling Tool (IMT) which is both a modelling and mapping tool to enable the automatic Information Exchange between Database and Message, Database to Database or Message to Message.
Summary and conclusion
Message Text Formats are carefully designed and controlled so that military users of formatted messages can be assured of receiving the timely and consistent information needed to make informed decisions. If the military user is a member of a multi-national force, then those formatted messages can be used to pass information that is understood by all recipients, irrespective of who or what generated them. One of the facets in the design of MTF is that messages should be capable of being read by computer. This enables messages to be prepared, validated, sent, received, parsed and read automatically using the power of modern computer and software systems. Thus, information contained within the message can be generated in one Command and Control System and read by another irrespective of the design of the database, operating system, GIS, or computing platform. True interoperability can be achieved without the need for identical systems in the field. However, some aspects, such as the MTFs and the distribution of the message rules to the operational end user, do have to be mandated and controlled. By using modern software tools such as IRIS this task is made much easier, as is the problem of having out-dated systems in the field.