The Status of Models in Systems Engineering

Introduction

This paper considers the nature of systems engineering (SE) and the role that models play within the discipline. It discusses how SE relies heavily on models to develop understanding and aid communication and decisions throughout its application. In SE these system considerations and modelling lead directly to a design for a purposeful system or process that is implemented in the real world. In the past SE has been associated with hard, well-bounded, well-precedented and (it seemed) well–understood problems. Through a consideration of the changing nature of modern systems and the difficulty that is observed in the early stages of the system design process we identify how approaches from other systems research disciplines are enabling modern SE to deliver complex systems and capabilities in the defence world.

What is systems engineering?

SE is the application of systems thinking to real-world systems problems in the field of engineering in order to achieve successful solutions to such problems. Engineering is generally considered to mean a process of problem solving, in that some set of objectives are achieved. It is defined as “… the profession of applying scientific principles to the design, construction and maintenance of engines, cars, machines, buildings, roads, electrical machines, communication systems, chemical plant and machinery or aircraft” [1]. Engineering is thus a discipline that aims to solve real-world problems through the application of scientific and technical solutions. Systems thinking involves the realisation that many of the things that we deal with in day-to-day existence can be considered to be systems; that is sets of entities related in some way, often to achieve some purpose. These systems can contain (any combination of) people, processes, technology, hardware, software and organisations. Thus the remit of SE is much broader than in the traditional mature engineering disciplines. Further, these systems generally contain sub-systems and are themselves part of wider systems. Indeed these systems, sub-systems and wider systems can be conceived in different ways and from different perspectives. When dealing with systems we need to consider issues such as boundaries, viewpoints and emergence—behaviour manifest at the system level that is not apparent at the sub-system level. SE can thus be seen to be the discipline that deals with designing systems composed of people, hardware, software, processes and procedures to meet user requirements, within a great variety of wider environmental influences.

SE is thus a systems discipline. As such, we need to consider a variety of systems consideration methodologies when looking at how to "do" SE. As stated above, we consider SE to be a discipline that applies the principles of systems to the practice of engineering. This is not a generally agreed definition. Many people see SE merely as a “systematic” process for the development of systems. Although, of course, process plays a large part in it, this view misses the “systemness” of the subject itself. Thus “traditional SE” is seen as being systematic, not systemic. It is often heard that “SE is a process not a discipline”. This leads people to regard SE as dominated by process and the production of “hard” outputs, such as formal documents and hard engineered products. In turn, this leads to an emphasis on corresponding “hard” methods and tools, such as information management tools and formal system design methodologies. We believe that in good SE the notion of system pervades both the process and the product. It is through a systemic consideration of the problem in its domain together with a systematic approach to its solution that designed systems that meet user requirements can be developed with confidence.

What is in a name? A key achievement of Peter Checkland is that he was able to coin an enduring phrase for his approach to problem solving in managed systems—Soft Systems Methodology (SSM). In the name SE we have a term that has existed for 50 years. In that time both the subject and the object have changed substantially; in essence, the words are the same, but the meaning has changed. Thus two people can have a conversation about SE without realising that they are talking about what are, in effect, different things. This is one of the problems that currently beset the discipline—“systems engineering” can mean all things to all men. We believe that the time is right for a re-evaluation of what is meant by SE.

The nature of modern systems challenges

The process view of SE is reinforced when one is dealing with well-precedented systems problems and solutions, so that SE becomes dominated by the systematic application of these processes, methods and tools to situations that are (apparently) so well understood that we do not need to focus much attention on explicitly developing an understanding of the systems problem, relationships, influences, inter-dependencies, and potential solutions. Further, SE has traditionally dealt with well-bounded systems such as platforms and so the need to build up an explicit understanding of different stakeholder views, wider systems and related systems has not been apparent - although, in fact, many of the problems that have been experienced when dealing with such apparently well-bounded problems have arisen because too many issues were taken for granted and not fully understood.

This whole situation is changing with the nature of modern systems challenges. Modern systems are highly integrated, complex amalgams of people, processes, hardware and software, where decision-making is often embedded and failure modes are far from simple and clear. They have multiple stakeholders and complex interrelations with other systems. These systems present us with much more open, unbounded, unprecedented problems (and opportunities) and demand a much more explicit systemic understanding of the systems problems, issues and solution to be developed and communicated. Models are key to the development of this understanding, its communication, and ultimately its realisation into a fieldable physical system.

Despite the fact that people have been engineering systems for thousands of years, SE is a relatively new discipline. Various standards have been issued over the last few decades, but it is only in the last 10 years that an international professional society has been set up to provide a forum for discussion of SE issues. In the UK there is no single body for the accreditation of SE courses. The latest work is represented by the development of an international standard in the discipline; ISO 15288 “Systems Engineering - System Life Cycle Processes” [2] is at the Final Committee Draft stage. This document identifies six stages in the SE life cycle. They are: Concept, Development, Production, Utilization, Support, and Retirement.

Figure 1 [2] illustrates these stages, and identifies the purpose that underlies it.

Traditionally, SE has been seen as a process that, systematically, enables a number of key phases in the development of systems solutions (such as development and production). These phases have been hard, well defined and clear, allowing the utilisation of well-defined engineering processes. For example Computer-Aided Design/ Manufacturing (CAD/CAM) modelling can allow the investigation of design concepts that clarifies the real-world ability to develop them. Simulation modelling can be used to inform and ultimately enable efficient manufacturing processes.

However the authors feel that SE should systematically and systemically enable all phases of the life cycle. It is important to consider all stages of the system life cycle at every stage, and this can only be done in a systemic way. Hence SE should be a systematic and systemic approach to problem solving and the design and integration of systems. Life-cycle phases such as concepts and utilization are less well defined, and demand a clear systemic consideration of requirements, related systems and the wider system of interest. This is critical because the whole SE endeavour (and in particular its success) is crucially dependent on the quality and robustness of the systems understanding on which decisions are made. Our view of SE is that it must address the development of this systemic understanding much more so than in the past may have been true. This expands the traditional, hard process view of SE and demands that SE tackles the "less well exposed" early phases of the SE life cycle in a much more explicit, creative way.

The SE endeavour can thus be seen as a process that allows initial understanding of requirements through the development of concept solutions, their evaluation, the making of trade-offs to selection to implementation. This process is in many ways characterised by the early stages, which represents an explosion of information and conflicting requirements before, through a process of refinement and the subsequent focussing of ideas and concepts, models are generated that can be used to drive a hard engineering process. The “Reaction Chamber Model” illustrated at Figure 2 illustrates this view of the early stages of the SE life cycle.

Exploration of these early stages of the system life cycle, reflected in the left-hand area of this model, requires a broader set of techniques than in the past, techniques that are generally not seen as being part of the systems engineer’s toolkit. Examples of such techniques that we have found to be useful include:

• SSM,
• Completeness and Whole System Modelling,
• Influence diagramming and qualitative system dynamics modelling,
• Mind mapping,
• N2 mapping, and
• Data modelling.

Traditional problem domain boundaries

There is great commonality between the methods of approaching problems applied by our variant of SE and soft OR (and, indeed, other disciplines where system modelling is fundamental). This is driven by our belief that the context of a (practical) discipline (such as SE) should be defined by the problem rather than a set of methods "developed" for the domain. To a certain extent, mature disciplines develop sets of recognised tools, methods and techniques through a process of evolution and over time. Different disciplines are bounded by the methods and tools that they use. In an evolving discipline this can be counter-productive. We believe that this approach emphasises differences that are not really there; the differences are in the applications (or problems) themselves rather than the methods. Thus SE is, to a certain extent, a method for looking at the world and trying to solve real world problems within it to meet requirements. We believe that a broad set of methods and tools is admissible in this quest. Indeed, we believe, rather as the early OR pioneers did, that no method is inadmissible if something is potentially to be gained from its use.

Systems engineers design and oversee the implementation of complex systems to meet varied and diverse user requirements. Operational researchers apply scientific methods to assist with decision making related to the operations of organisations. Both disciplines are about making decisions in complex environments. Both disciplines rely on simplifying initial complexity in order to gain insights into the way systems are organised, and, in both disciplines models are central to the problem solving approach. In SE it is through the use of models that user requirements are exposed, that concepts are visualised and, ultimately (through design documents) that systems are built. In OR it is through the development of models and their investigation that we draw conclusions about how to behave in the real world.

The uses of models

We can characterise models according to the use to which they are put. It is arguable that all modelling is done in the light of a problem and to drive behaviour or action—else why conduct the modelling activity? Yet models may generate many things—“answers”, common understanding, insight, and so on. An often-used taxonomy for describing the use of models is to consider them as being:

• Descriptive. Models that explain or describe a problem, phenomena or system. An example of a model used in a descriptive sense might be an organisation chart. Such a model is useful for system understanding and communication.
• Prescriptive. Models that indicate courses of action that are in line with our requirements. An example of a model used in a prescriptive sense might be a linear programming or optimisation model designed to inform something like factory throughput.
• Predictive. Models that indicate how the world may evolve in the light of certain decisions or actions. An example of a model used in a predictive sense might be a wargame designed to illustrate the consequences of particular combat options or force mix decisions.

However, we do not feel that this typology is particularly clear or that models belong to a single one of these types. Rather we see that, depending on why they are being developed and on where, when and how they are being used, the use to which models are put will emphasise different aspects of each. Consider, for example, the use of an Ordnance Survey map. This map is a model of some piece of terrain that exists in the real world. It may be used descriptively to aid the development of understanding of the nature of that piece of terrain, its geographical features and attributes. It may be used prescriptively to allow us to select a particular route between A and B. Finally, it may be used predictively to allow us to forecast the likely implications of our actions; for example, if I continue on this bearing for this time at this speed I will arrive at C.

However, even between the disciplines of OR and SE there are subtle differences in how these three terms are understood. In mature engineering disciplines, such as civil, mechanical, electrical or electronic engineering, "models usually meet the criteria for hard models. (These) models draw on the theories of the natural sciences and engineering to define key attributes and their interrelations, and use an internationally agreed measurement system as the basis of characterizing attributes. Armed with such a model, the engineer can describe the process, set standards by prescribing the attributes which the product or process must manifest, and predict the output of a future system from the input and the process." [3] However, there is little in the mature engineering literature that addresses the methods by which “soft” issues can be addressed and communicated in a rigorous way. We see a clear requirement for methods, tools and notations that allow the consideration of these issues and thus enable a continuum with the hard processes that must follow in developing tangible, engineered, robust, well proven products.

We see the triad of description, prescription and prediction as being central to the utility of modelling in both engineering and OR. However, there is perhaps more in the OR and general modelling literature about these three uses of models. We expand on these uses below.

Models used descriptively

Models used in an explanatory or descriptive fashion generally help us to understand something. As Casti has said, "the primary purpose of such a model is not to predict the future behaviour of a system, but rather to provide a framework in which past observations can be understood as part of an overall process. Probably the most famous model of this type is Darwin's Principle of Natural Selection, by which one can explain the appearance and disappearance of the many types of living things that have populated the Earth over the past four billion years or so ". [4] Such models "serve well to explain what has been observed .... by providing an overarching structure into which we can comfortably fit many known facts. However, when it comes to predicting ..(anything).. these models remain silent". [4] This type of descriptive model is hence great for generating insights, for communication and for increasing understanding. They may even lead to an awareness of causality and dynamics. But they do not tell us how to behave. To a certain extent they can be seen as the first step on the modelling route: the development of insight and understanding in order to allow us to generate the confidence to use models in a prescriptive or predictive sense. As such they are extremely useful in the early stages of problem exploration, where the nature of the problem is not agreed and the nature of the system is not clear.

Models used prescriptively

Prescriptive models tell us how to behave. More formally, based on some assumptions or observations of the real world, a model used in a prescriptive sense will indicate how best we can meet our goals. "A prescriptive model specifies a course of action. Linear programming, dynamic programming, game theory and decision theory are methodologies that solve problems in ways that tell one what to do”. [5] It is important to note that this necessarily involves drawing conclusions from the model. Thus the process of using a model in a prescriptive sense involves learning about the model and extrapolating to the real world.

Models used predictively

Models used in a predictive sense indicate how the world may evolve. As Casti says, "Newton's model for the motion of gravitating bodies is an example of what is called a predictive model. Such a model enables us to predict what a system's behaviour will be like in the future on the basis of the properties of the system's components and their current behaviour". [4] Thus it seems that, as with models used in a prescriptive sense, when models are used in the predictive sense we are discovering what the implications of the model are and drawing conclusions about the real world. In actual fact, all we get to learn about is the model world. Casti goes on to discuss this later. Talking about the solution of a mathematical five-body system, Casti states that the solution "solves only a mathematical version of the real-world problem; what it says about a real five-body problem is anyone's guess". [4] There is thus much overlap between what is meant by predictive and prescriptive in modelling.

Hence it seems that models can only be used predictively or prescriptively when the user has a good degree of confidence in the assumptions underlying the model (or the theory). The development of these assumptions or theory is accomplished in the initial use of models in a descriptive, explanatory sense. As has been stated (with particular relevance to models used in military applications, "it is doubtful that there are predictive models which are entirely distinctive from descriptive or prescriptive models. However, we want models with predictive power. When we are satisfied that a model describes an existing situation or phenomenon adequately, then we want to apply it to process input data and arrive at results in other situations". [5] This is the nature of simulation in both SE and OR. We wish to develop models that can help us address the world that we don’t yet know with confidence.

The status of models in systems engineering

As we have stated above, we do not believe that this forms a mutually exclusive taxonomy. Rather, for some particular situation, we believe that the actual model is used is in a way that is a combination of descriptive, prescriptive and predictive. We have hence found it helpful to view these as being the vertices of a triangle. Figure 3 illustrates this idea. This simple visualisation allows us to ask a number of questions about the development of understanding as the SE endeavour proceeds. Initially, the problem is unclear, user requirements are unclear and we have little understanding on which to base firm “hard” engineering models. We develop an understanding of problem, stakeholders and requirements through iterative development of soft models used in the descriptive sense. We have found the techniques identified above particularly useful at this stage. As understanding evolves, we visualise progress from the left-hand vertex towards the right-hand leading edge. We can thus, in a simplistic sense (and very much in line with the idea of a simple model being used for descriptive purposes) view the SE “journey” as one from the left-hand vertex of the triangle to the leading edge.

Thus the initial stages of the SE endeavour rely on soft models for the description of the problem domain and the world within which it exists. Thus, models are used as transitional objects that allow the development of agreed problem statements and requirement sets, allowing the development of potential concept solutions. These concept solution models can be used in a progressively prescriptive and predictive sense to analyse how their introduction into the real world problem domain might meet requirements (or not). Evolving understanding is reflected in improved descriptive models in which users have greater confidence and hence their predictive and prescriptive use. This process is illustrated by the annotated DPP Triangle in Figure 4.

We can thus see the learning process as one where the development of models leads to the generation of real-world understanding in an iterative sense. This is reflected in Figure 5. We generate descriptive models reflecting our assumptions and understanding of some situation, and generate models to be used prescriptively and predictively based on that understanding. The output of theses models will further allow the refinement of our understanding and hence our descriptive models. There is thus an iterative, feedback relationship which captures the transition from initial system observance and understanding to the introduction of “solution systems” that users believe will fill the required gap based on their understanding of the system. It is the initial stages of this iterative process that has been conducted informally (if at all) in traditional SE and which we believe needs to be clearly recognised in modern SE.

Validation is the process through which we evaluate the consequences of the model against real-world observations. This brings out the essential difference between modelling and simulation. If we define modelling as the development of a representation of some aspect of the real world in some context, then we can see simulation as the evaluation of the consequences of the model. Thus (at least in time-based models) simulation generally generates behavioural information from structural information.

In engineering terms, this can be construed as the development of initial “high-level” designs and the investigation of their implications through an iterative process of engineering design until we arrive at a design which we believe meets the requirements of users and is realisable in the real world. Thus we can see how the SE endeavour leads from an initial development of understanding through the use of descriptive models, through the development of designs and specifications and their assessment through simulation to the realisation of a physical system solution. As part of this endeavour, the key “outputs” that any engineering process requires (such as User Requirements Documents (URD), System Requirements Documents (SRD), acceptance criteria and detailed design documents emerge as a consequence of the systemic consideration.

Conclusions

We feel that much of traditional SE involved a process for translating clear and unambiguous requirements statements into physical systems. And it had great success in delivering systems where ambiguity was not present. However, wherever we are unclear as to the nature of the systems problem or stakeholder requirements we feel that we need a broad set of tools to help us develop descriptive models that we can confidently use as the basis for our development of prescriptive and predictive models. We believe that a number of the methodologies that have been developed in other areas of systems investigation are useful for this. It is only through a comprehensive development of understanding that we can be sure that our engineering designs (our prescriptive models) will suit our requirements. Thus we advocate a broad use of these techniques in the early stage of the SE life cycle in order to improve the chances of a system that meets user requirements being delivered. Such a view takes SE far beyond its traditional “hard” boundaries and recognises the changing nature of modern systems, expectations and environments. In particular, we believe it is essential for SE to develop in this way in order to tackle the increasing, complexity, integration and uncertainty of the modern “systemic” world.

References

Authors

Sean Price is a lecturer in the Engineering Systems Department (Applied Mathematics and Operational Research and Systems Engineering Groups), Cranfield University at the Royal Military College of Science. He is the Academic Leader of the Systems Engineering for Defence MSc. Prior to joining the academic staff of RMCS, Sean was an officer in the Army for 13 years, leaving as a Major in 1998.

Philip John is the QinetiQ Professor of Systems Engineering in the Engineering Systems Department, Cranfield University at the Royal Military College of Science. He spent 18 years in the defence industry before joining Cranfield University in 1999. He leads the growing Systems Engineering Group. He is a member of several national bodies, including two National Advisory Committees (for Systems Engineering and for Synthetic Environments), the MOD’s Smart Requirements Review Board and the IEE’s Executive Team on Systems Engineering.