(Still) Striving for Utopia

Introduction

About 30 years ago, as a young Air Force engineer, the author of this paper was called back from a great posting in Singapore to participate in the rebirth of a major project that had gone belly-up. This massive ‘people and software-intensive’ project called LINESMAN was being developed to provide air defence coverage of the UK airspace. It had run into all sorts of problems not the least of which was that the software didn’t work, principally because of totally unrealistic operational requirements. For example, the software was supposed to detect all situations requiring operator action, warn the appropriate operators and automatically route all relevant radar track data to them. In addition the software was to ensure operators were equally loaded at each position on the assumption that all sixty or so of them were equally capable, which was clearly not the case. Furthermore the software design was to enable command AND centralised control to be exercised over all military air defence movements in the one million square mile box of the UK air space. Not unsurprisingly it had become impossible to elicit the mountain of logical rules covering these operator requirements. So all work on the software and the project was stopped.

Following a one-year hiatus, major simplifications of the requirement were made to permit operators to control their own radar track management workload, whilst aircraft interception was removed from the centre and delegated to the sector radar sites dotted about the country, thereby relieving the software of these onerous tasks. By reducing the software task to one of picture compilation and putting the operators in control, three years later a new and much simplified LINESMAN project was up and running very successfully.

All of this took place early in the author’s career and formed a first class introduction to the potential pitfalls that can occur in the pursuit of the automation of large-scale man-machine information systems. The experience was destined to change his career and working life. This paper develops some thoughts and concerns on the same subject today.

Clumsy automation

The ability of the computer to perform lengthy and complex calculations as well as store and manipulate vast quantities of data with great speed, accuracy and reliability (tasks at which, it is important to note, humans are not generally good) has led the computer to be described as a sort of intellectual technology capable of extending our brains and minds rather than just our limbs and muscles. In those early years, the structures of many discrete, repetitive operator and clerical processes were well understood. As a consequence they could be analysed using logical analysis techniques and so could be successfully automated. Straightforward benefits in terms of cost savings through replacement of human labour with software were quickly gained. Even so, significant mismatches between users, operators and the software were all too common (as they still are). This period has been referred to as the era of CLUMSY automation. Systems worked but sometimes not very well and often not to the operator and user’s satisfaction.

Hazardous automation

The last two decades have seen rapid development and application of Information Technology (IT) across the board. Computing power is doubling per unit cost every 18 months. Nearly unlimited and cheap data storage and communication bandwidth is with us with more to come. Client-server architecture is giving way to networked and mobile collaborative computing, and the limits imposed by physics on hardware developments are still some way off. So in the years ahead we can anticipate near-infinite electronic space, which we can fill with yet more software and data if we so choose. ‘Parkinsonian’ tendencies to fill all of this electronic space with data and software will be rife.

But what will we then do with all of this increasing-large electronic data mountain? How will human organisations interrelate with these burgeoning software structures? We should do well to recognise that the human condition and our ‘bandwidth and processing capacity’ (that is, our IQ, cognitive and perceptive abilities and so on), as well as our organisational, social, cultural (even anthropological) developments, have been very slow to change. They are rooted in the data-poor culture of yesteryear. We have not kept pace with the dizzy rate of change and developments in IT. Indeed, with the rapidly diverging gap between the human condition and this kind of technology, there is a growing concern that our brains, organisation structures, consciousness and even our general well-being could become increasingly overwhelmed. And because software automation is not yet powerful nor trusted enough to bridge successfully the widening chasm between the two, particularly in some high performance time-critical man-machine applications, there is a rising concern that future systems might under-perform and alienate users, sometimes with unfortunate consequences.

This must be a worry for many organisations, including the military, because they are under relentless cost-reduction pressures from politicians, top-management and manufacturers to introduce IT across the enterprise. This strategy forces the ‘integration’ (some would say ‘graunching’ (Air Force slang)) of business processes, the fusion of large quantities of disparate data, and the restructuring and interconnection of organisations. And because these changes are often attempted concurrently and sometimes ad hoc, systemic problems regularly occur and widespread user dissatisfaction can result. Clearly the logical systems analysis techniques, so successfully employed in the early automation of rationally bounded clerical tasks physical control problems, are now unable to capture the predominantly unstructured nature of the complex human and business interactions of these new ‘business’ situations. It seems evident that the slavish pursuit of complex system solutions using orthodox algorithmic thinking alone will result in major discontinuities being experienced between the end-user organisation and the projected automation, just as occurred in the LINESMAN project mentioned in the introduction to this paper.

The management and control of this emerging state of affairs invites an understanding of, and answers to, two very important questions. What can be successfully automated and what cannot? What should be automated and what should not? The first considers what is technically feasible, within the set constraints. The second asks, even if technically feasible, there may be sound reasons for limiting the automation.

Many of the problems in matching digital automation with organisations and human beings have been well chronicled. Difficulties are not confined to the military. In the world of commerce, research suggests that many business re-engineering projects in the mid-90s failed to perform because of an over concentration on the analysis of ‘process’ and the technical solution. Often end-user and organisational issues were not given the same weight as technological ones. Users and operators were given little opportunity to influence the design. So you will not be surprised to learn (you might know already) that the creator of the term ‘business process re-engineering’ was not a user, not an operator nor a management buff but a computer scientist. The term ‘business re-engineering’ seems to be a very good example of an oxymoron.

There is insufficient space here to list the many complex IT-based projects which have failed. But one is of particular relevance to the theme of this paper . In 1988, in the Straits of Hormuz, the USS Vincennes cruiser, a ship equipped with the very latest AEGIS automatic air defence command and control system, shot down an Iranian Air Bus with great loss of life. After the catastrophe had happened, computer records from the ship showed that the technology had worked as designed. But clearly the system had not. A combination of factors had resulted in cumulative stress building up in the whole operations room team and a false mental model of the outside world being formed by the command. The result was that fatal firing of the Standard surface to air missile.

Rochlin [1] in his authoritative assessment of the incident described the tragedy as an example of a new class of BIG SYSTEM problem to which military and other large organisations could become increasingly susceptible as highly complex digital automation is introduced into operation in the changing world. He makes the most important observation that the complexity of the AEGIS technology forced the crew into a control and not a management mode of operation. He says that the system had scenarios programmed into it for air defence of a carrier battle group in a total war operation. It was not designed for operation in the gun-boat infested waters of the Gulf in which there was considerable civil air and sea movements. It was not therefore surprising that the crew so extensively trained and practised on simulators which played the games for which the system was originally designed, should fall back into one of these many pre-programmed behavioural patterns when they were put under severe operational stress and time pressures.

Rochlin distinguishes between the concepts of control and management by saying that, if one has a perfect understanding, correct information and a verified knowledge base that encompasses all possible future variations in the target environment, then one can indeed delegate full control to software machines and hence automate all possible outcomes. However, management is, inter alia, where control is delegated to people and not machines. It is the necessary alternative decision-making regime when the designer is faced with irreducible operational uncertainty, even unknowability, and where flexibility is required, such as is common in the context of emerging military and other potentially adverse environments. Given the uncertainty of the world in which we now live and the insatiable demand for software automation and networked applications, we could now be entering a period of what has been called HAZARDOUS automation (with all the potential political, societal and military fallout which might result) instead of the clumsy yet potentially safer and more benign automation of yesteryear.

It is suggested there would be much merit in calling these new systems socio-technical systems (and not equipment systems) because, though an ungainly term, it does capture the necessary coexistence of two essentially very different sub-systems, the socio and technical sub-systems, both of which need to be considered together in the design of the solution as a whole. The term is not new. It arose from the operations research studies performed by Trist and Bamforth [2] at the Tavistock Institute in Belsize Park, North London when the installation of automatic coal cutting equipment radically changed the social and work organisation of British coal mines in the late 1940s.

The new utopians

A little before the difficulties with the LINESMAN project, a book The New Utopians was published in 1965 written by Robert Boguslaw [3] whilst at the RAND corporation in the US. At the time, he was engaged on development of the (successful) US air defence equivalent of the (then unsuccessful) LINESMAN system in the UK. His writings were prescient.

In the book, Boguslaw opines that computer programmers actually prescribe the patterns of social behaviour permitted by users and operators of computer-based systems. The programmers are concerned not with people but with what he calls people substitutes. The programmers are concerned with neither souls nor stomachs. They are the social engineers of our time. He states the systems they aspire to design have all the characteristics of classical Utopias because they strive to achieve operating modes free from human imperfection and, ergo, human involvement. He calls these engineers ‘the New Utopians’ which goes some way to explaining the title of this paper.

Fundamentals

So why today do we still have concerns and difficulties with our large man-machine systems when most other large-scale, but predominantly physical, projects such as roads, bridges and buildings seem to complete routinely to time and cost and to the User’s satisfaction. Is it because the construction of physical objects is well supported by an established and validated science base (for example physics, mathematics and electromagnetic theory) which then underpins the relevant engineering disciplines (for example civil, mechanical and electrical engineering)? Use of the methods and tools from these established fields of engineering then ensures the risks in design and construction are low and manageable. Moreover, with an advanced education and training system in place, sufficient numbers of well-qualified and skilled engineers are available to ensure a high chance of success.

Compare this with the situation for the kind of systems that are the subject this paper. Firstly there is no consensus as to what forms the appropriate science base. It is certainly interdisciplinary and comprises specialist sciences of quite different characteristics ranging from sociology, psychology and linguistics to data communications theory, physics and mathematics. Secondly, with no common interdisciplinary language between the disciplines, there is very poor communication between the practitioners. Significantly, since the sociological, physiological and psychological disciplines are as important as the mechanical, electrical and software engineering ones in overall system design, you might question whether we should continue to use the traditional concept of engineering (with all its intellectual baggage of calculative rationality) in the context of these soft and very people-intensive systems.

So, with this deficiency of inter-disciplinary tools and the technology rattling ahead, the increasingly optimistic benefit expectations of some of our users could be running ahead of our ability to deliver services to acceptable levels of cost and user satisfaction. We all could be much at risk.

The views of Devlin [4] in his recent book ‘Goodbye Descartes’ are of interest. He identifies man’s need for two types of tool. The first are the technical tools. These are the technical artefacts such as computers and communications equipment. The second are more fundamental tools, such as science and engineering which he calls conceptual tools. Conceptual tools provide us with the intellectual mechanisms for applying the technical tools successfully. He says we are rapidly reaching the limits of understanding of what can be achieved through the use of the traditional conceptual tool kit based on physics and mathematics. Devlin says we need a new conceptual tool kit, a systems science tool kit if you will, which admits to the inclusion of people and context, concepts which are alien to and excluded from the traditional tool kit.

Indeed in the 1970s Sutherland [5] in his seminal paper ‘Systems Science and Social Integrity’ observed that the design of emergent complex socio-technical systems was an example of a new class of meta-problem. He observed that the resolution of these meta-problems required a ‘next generation’ methodology to conjoin the hard quantitative and soft qualitative sciences. He urged we be cautious about letting traditional specialists loose on this kind of problem because there was a danger they would reduce the meta-problems to what was tractable and analysable. If this was permitted to happen, we should effectively have passed the resolution of the (meta) problems to the very people least qualified to deal with them. As a consequence system integrity would always continue to be an elusive goal. Although speaking in a general vein some twenty-odd years ago, Sutherland’s plea for a systemic understanding and need for interdisciplinary tools and skills is of widespread relevance today.

There is now a resurgence of interest in ‘systems’ matters, particularly in the engineering movement. One must applaud those who are currently pushing the systems engineering approach and methodology because clearly something along these lines is needed. But there are dangers ahead. At the risk of receiving much ‘flack’ (but hopefully to spark off some debate on the subject), the author cannot help but feel that a lot of what passes for ‘systems engineering’ today is in effect hardware and software engineering re-badged as systems engineering. Much lip-service is being paid: a liberal smattering of words such as ‘user’, ‘man-machine interface’, ‘human factors’ and so on is evident. But other ‘socio’ matters such as context, unpredictability, working environment, competition, purpose, capability, output, business transformation, organisational restructuring, personnel selection, training, job satisfaction etc and importantly the links between them and the technical aspects, are not there. Leafing through these tomes, one cannot help be left with the impression that this orthodox view of systems engineering is just too equipment-centred. It has been created for traditional engineers by traditional engineers and falls well short of what is needed in the future. The renewed interest in systems engineering is to be welcomed but it does need to be developed along socio-technical lines.

But getting back to fundamentals, we really should be concerned about the lack of some form of cohesive systems science base. If one began to emerge, it could only serve to strengthen the development of a suitable systems-engineering discipline. Hopefully readers will tell me that the necessary research and study is underway already. But if this is not the case, then perhaps investment should be put in hand to develop the much-needed conceptual tools.

Conclusion

So, in conclusion, it could be said that computer programmers (along with the business process analysts) still reign as the social engineers of our time; they are still the New Utopians, as Robert Boguslaw called them way back in 1965. But there are worrying signs that the systems engineering movement itself, probably unwittingly, is showing Utopian tendencies. Its culture is still rooted in the predominantly ‘Cartesian’ logical and rational model of the world, a model that does not admit to the inclusion of people and context.

What should be done to avoid inappropriate technological determinism in these types of system? Well, in the short term one could do a lot worse than put engineers with social and human factors skills to control, or strongly influence, the system design process. But in the medium to longer term I believe a more substantive systems science base is needed. For without such a base, systems engineering will be grossly deficient and increasingly there will be ever-present risks that our solutions will fail in extremis, sometimes with serious consequences. Then, instead of this most exciting of new technologies bringing enhanced social and military benefit, and liberating our minds, we could end up being the slave of our own creation and not its master.

References

Author

Malcolm H. Mills, BSc, CEng, FIEE, is a graduate of the Royal Military College of Science, Shrivenham and London University with 30 years experience in the acquisition and logistic support of computer-based systems for the Military. Following a General List career in air defence, avionics and software management in the engineering branch of the Royal Air Force, he entered the UK Civil Service Science Group as a MoD(PE) project manager of Naval shipborne, combat and NATO tactical data exchange systems before leaving to join the IBM Defence business in 1982. To pursue further his interests in User issues,in January 2000, he joined Gregory Harland Ltd of Windsor, a leading consultancy in organisation development and the human sciences.