Experimental Injury: Inference From Proxy Observations In A Test To The Real-World Average

Introduction

Consider the following. (A) A test is conducted of something that may cause injury, or of something that may protect against injury, and some physical measurement is made during the test. (B) Of chief interest is the average degree of injury that will occur in real-world conditions. There is a considerable gap between the test (A) and the real world (B). This paper is about closing that gap—that is, saying something about real-world performance on the basis of what is observed in a test.

The two contexts used as examples in this paper both refer to injury and injury protection.

• It may be desired to start from a small number of tests in controlled conditions of a specified missile into animal tissue or a tissue simulant, and extrapolate the observations to the average effect for that missile over the whole human body. Kokinakis and Sperrazza [1] specified such a procedure. They obtained the average probability—averaged, that is, over the different parts of the body—of a soldier being incapacitated by the missile.
• Tests are routinely conducted where instrumented headforms are projected at the fronts of cars [2], in order to check that the car is not overly injurious to a pedestrian. These are at one speed, but better information would be obtained by averaging the performance over the range of speeds occurring in practice [3]. Here, speed takes the role that part of the body does in Kokinakis and Sperrazza’s procedure—it varies in the real world and is important to the outcome, but the test does not reflect this variation.

Payne [4] has recently emphasised the importance of experimental and observational studies of wound ballistics. There is specific discussion of the average wounding potential of a random hit on a soldier. The present paper concentrates on the concepts average and random. It will describe the Kokinakis-Sperrazza procedure very differently from the account in [1]. The reason for doing this is that the basic issue is that of inferring real-world performance on the basis of a test result in one set of conditions—surely a common problem, and easier to appreciate if the formulation is in general terms.

The next section will identify the information that is needed and how it is put together. This is then illustrated by describing the procedure of Kokinakis and Sperrazza. Another example of the same general idea is then given, in the context of testing of blunt impact using instrumented headforms. Calculations are presented that both illustrate the method and make a substantive point about the danger of cushioning being too soft. Finally there is a discussion section.

How testing relates to real-world performance

One purpose of testing is to predict real-world performance. Conditions in the real world often vary enormously. Testing might attempt to reflect that variation. However, it seems to the present writer that, instead, testing becomes dominated by a single set (or a few sets) of conditions that can be specified in a written protocol and conveniently achieved in a laboratory test setting. The aim in this paper is to describe a series of calculations that to some extent will compensate for the limited variation practicable in testing.

In short, testing is conducted in specified conditions, whereas conditions in the real world are highly variable. Popular strategies to resolve this dilemma include the following. (a) Test conditions might be chosen that are typical or representative of real conditions. (b) Conditions might be chosen that are harsher (more hostile) than real conditions, as in “accelerated life testing” of equipment. (c) Rather than either of these strategies, it might seem better to test in several different sets of conditions, in order to reflect real-world variability—but the cost of testing puts a limit on this.

This paper will consider a fourth strategy, that of using the result of one test to calculate average real-world performance. There are three steps.

• From what happens in a single test, generalise to what would happen in a test in any other set of conditions.
• Convert the result of a test into a “value” or “cost” or “utility” of that result—that is, into something that can be averaged.
• Use the probabilities in the real world of the various sets of conditions to calculate the average real-world value corresponding to a test result.

The method is thus one of generalisation, valuation (or costing), and averaging.

The procedure of Kokinakis and Sperrazza

Description

Averaging will be required, and so it is appropriate to use mathematical notation. Some of the symbols, however, do represent names rather than numbers.

The building blocks are the quantity x that varies in the real world but is constant T in the test, and the functions h(T) = the result of the test, p(h) = the value or cost of the result h, and f(x) = the proportionate frequency with which x occurs in the real world. The meaning of this notation in the context of [1] is as follows.

• h(T) represents the wound in the test T (h is a name rather than a number).
• h(x) represents the wound when the soldier is struck at point x (and both x and h are names).
• f(x) is the proportion of the area of the body that the point x represents. All parts of the body are assumed to be hit with equal probabilities.
• p(h) is the level of incapacitation from wound h. This is 0, 25, 50, 75, or 100 per cent: p is a number, and can be averaged. It depends on the tactical situation and the time after wounding as well as on the wound.

In test protocols as commonly conducted, the test result, h(T), is of direct interest: for example, there may be some protective clothing or armour at the wound site, and a minor wound leads to passing the test and a major wound leads to failing. However, in the procedure of Kokinakis and Sperrazza, the wound in the test is instead evidence for what the function h(x) might be. The following is from pp. 9–10 of [1]: “For each combination of missile and striking velocity the procedure for establishing the size (depth and lateral extent) of a hypothetical wound tract in a soldier first involves firing the missile into different types of anatomical components of goats.... These data serve as inputs to medical assessors who then estimate physiological effects in soldiers subjected to the many hypothetical wounds that could arise from random strikes.” Thus interpretation and extrapolation to other parts of the body are judgments on the basis of medical expertise, as is also the construction of a table giving the estimate of incapacitation p from any specified wound h.

• Extrapolation to other parts of the body is the process of inferring h(x), the wound when struck at any point x, from the observed h(T), the wound observed in a laboratory test.
• Estimation of incapacitation is the process of using a previously-constructed table that specifies level of incapacitation p corresponding to any wound h.
• p is determined by h, and h is determined by x, and this is written as p(h(x)).
• As p is number, it can be averaged:

where the summation is over all the different parts of the body, x. Although the set of probabilities f(x) is said to refer to the proportions of the area of the body, with all parts of the body being assumed to be hit with equal probabilities, the concept of area of the body is not really needed, as it can simply be said that f(x) means the proportion of cases of a particular type denoted x.

Equation (1) is a perfectly ordinary averaging equation: p is a score reflecting how much incapacitation there is, f is the fraction of occasions on which it occurs, and Avp is the average of p. It should be admitted that each of the steps is demanding of data, in the sense of it being difficult to make reasonable estimates of what the functions h(x), p(h), and f(x) might be. Some types of decision, though, may not be greatly affected by some types of error: for example, comparison of the implications of two test results (that is, which result implies the higher Avp) is not affected if all the p’s are in error by the same percentage.

The final result is specific to the type of missile, speed of missile, time after wounding, and the tactical situation (which refers to attack, defence, reserve, or supply). Type and speed of missile affect the wound, and time after wounding and the tactical situation affect what level of incapacitation results from the wound. There ought, perhaps, to be averaging over different angles of impact, as well as over different locations: this is mentioned by Kokinakis and Sperrazza as a desirable refinement. And the cost function p(h) is likely to itself reflect one stage of aggregation already: a given wound may have different effects on different people, and p(h) represents an average.

An example at page 18 of the report by Kokinakis and Sperrazza runs as follows.

• This refers to a particular type of missile at a particular speed, to a soldier in an assault role, and to a post-wounding time of five minutes.
• For a fraction 0.414 of the body area, a hit is considered to result in zero incapacitation. A hit on no part of the body is considered to result in 25% incapacitation. For a fraction 0.212 of the body area, a hit is considered to result in 50% incapacitation. For a fraction 0.210 of the body area, a hit is considered to result in 75% incapacitation. For a fraction 0.163 of the body area, a hit is considered to result in 100% incapacitation.
• The average is thus 0.414×0 + 0.000×25 + 0.212×50 + 0.210×75 + 0.163×100 = 43% incapacitation.
• What is missing from the above is a description of how the fractions 0.414, 0.000, 0.212, 0.210, and 0.163 were found. These came from a medical assessor’s interpretation of the wound in the test, extrapolation of this to other parts of the body, and (what is, in principle, a separate process of) estimation of incapacitation from the specified wound in the specified circumstances.

Further example calculations—showing the whole chain, from x to h and f, from h to p, and from f and p to Avp—will be given later (Tables 2-5).

Functional group

It is worth noting that Kokinakis and Sperrazza make considerable use of the concept of “functional group” or “disability group”. In the present interpretation, this is a special aspect of the context they are concerned with, wounds and incapacitation, and is not a core feature of the procedure. It is a natural way of organising the great variety of wounds that might possibly occur. In addition, as regards the overall structure of the general procedure, the relevant features are as follows.

• The effect of wound on level of incapacitation is captured completely by the disability group that the wound falls within.
• Level of incapacitation depends on both tactical situation and disability group.
• The disability group depends on post-wounding time as well as on wound.

(The dependencies referred to are detailed in Tables III and V of the report by Kokinakis and Sperrazza.) Let d represent disability group, t represent time, and s represent tactical situation. Then d depends on both h and t, and p depends on both d and s. Thus p may be written as p(d(h(x),t),s). However, d, t, and s do not need to be shown, as Equation (1) is evaluated after post-wounding time and tactical situation have been specified (and so dependence on t and s can be omitted), and d serves only to group h (and thus is superfluous also). The result is p(h(x)).

Similarly, Neades [5] distinguishes between injury, impairment, and effect on performance. In the present interpretation, what Neades refers to as injury is an observation in a test, and (for example) may closely resemble human injury if the test is carried out on animal tissue or may be a measurement of penetration if the test is carried out on gelatin; effect on performance is the required output, and must be a number as it will be averaged; and the concept of impairment (in this case, an intermediate concept) may not be strictly necessary.

Commentary

The above is not a complete summary of the report [1]. Other matters dealt with there include, for example, the proposition that the probability of a soldier being incapacitated when struck by a flechette of mass m and velocity v is more closely related to m.v^1.5 than to m.v or m.v², and equations for this probability. The present writer has the impression that the report [1] has been influential as regards the quantitative effects of m and v, and as regards disability group and incapacitation, but relatively little attention has been paid to the concept of starting from a test result and then generalising, valuing, and averaging.

A key element of Equation (1), in the present interpretation, is the process of averaging: the end result at a population level depends not only on what is likely to happen to a soldier (as represented by h and p) who sustains an input x, but also on what are the x’s received by different soldiers (that is, the probabilities f(x)). There is nothing similar to Equation (1) in the report, but there certainly is a process of averaging, as in the example given earlier. The absence of Equation (1) can probably be attributed to T, x, and h being names or descriptions rather than numbers, and so it may not have occurred to Kokinakis and Sperrazza to use mathematical notation and write the function h(x) for obtaining h from x.

Starks [6] and Neades et al [7] build on the work of Kokinakis and Sperrazza, though both these reports rather neglect variability in x and the consequent averaging. Neades et al describe part of the analytical crew casualty assessment process as follows. “(1) Assemble the parameters that quantify a battlefield insult. (2) Evaluate the anatomical / physiological injury produced by that insult. (3) Relate that injury to the attendant impairment, expressed as a degradation of elemental capabilities.” That is, decide what x represents, find h(x), and find p(h). The present paper shows how this fits between a test result and real-world performance: the test result helps to specify h(x), and real-world performance is calculated by averaging over the different values of x, using data on their respective probabilities (that is, f(x)).

Reports by Eberius and Gillich [8,9] and VanAmburg [10] consider variability in the input x.

• The following is from [8]: “Modelling the operational scenario in which the body armour is exposed is most relevant; this applies a weighting function to the protection offered by the system and supports risk analysis for fielding decisions”. Mention of “operational scenario” implicitly emphasises the importance of f(x).
• In [9], the likely level of injury (corresponding conceptually to d in the present terminology) is shown at different points on the face for helmets worn correctly and incorrectly.
• In [10], the likely level of injury is estimated at different impact points of a fragment on the limbs or face. The calculation is then taken a step further, to give a probability over a random point of impact of a specified level of injury. In the present terminology, p(h(x)) is 1 if the AIS (Abbreviated Injury Scale) score corresponding to h(x) is at least 3, and is 0 otherwise, and the probabilities of different impact locations f(x) are assumed equal

Table 1 describes what x, h(x), p(h), and f(x) mean in the context of [1] and in the context of headform impact tests (now to be described).

Tests using instrumented headforms

Equation (1) is intended to be a clear method of showing how the likely real-world performance is calculated from a test result. In principle, this is relevant to a wide range of tests of people and equipment. It was developed in the context of pedestrian impact testing [3]. In that context, the main concern is with blunt injury, not penetrative injury from missiles.

Speed x is the most important quantity that varies between different pedestrian accidents; x is a number, rather than a name.

• f(x) is the proportion of impacts that are at speed x. This is obtained by studying real pedestrian accidents.
• h is a number also, a quantity termed the Head Injury Criterion (HIC) that is calculated from measurements by accelerometers inside the headform. The headform is projected, according to a test protocol that includes specification of the speed, at various points on the front of a car.
• A function h(x) can be assumed based on a test at one value of x plus some theory of forces in impacts, or from testing at several different values of x.
• The function p(h) ought to represent the “cost” or “utility” associated with a particular level of HIC. In practice, it is more likely to be the probability of death or the probability of serious injury associated with a particular level of HIC. It is obtained by such methods as conducting experimental impacts of cadaver heads, observing both the accelerations and the injuries.
• Once f(x), h(x), and p(h) have been specified, the average is calculated as in Equation (1), where the summation is now over all speeds, x.

In this context, T, x, and h are numbers rather than names, and it may be possible to use theory to construct h(x) from h(T). This is rather different from the missile penetration example, in which much medical expertise is needed to estimate human wounds from animal experimentation.

Comparison of performances

Testing is often intended to permit comparison of performances (for example, of alternative designs of hardware). It may be that design A is better than design B in one set of conditions, but B is better than A in another set of conditions. (This might be found empirically, or predicted theoretically.) The possibility of this brings home the need to have good estimates of (a) the dependence of performance on conditions (h_A(x) and h_B(x) for the two designs, in an obvious notation), (b) how much the difference in performance matters (that is, the difference p(h_A)–p(h_B)), and (c) how likely the different sets of conditions are (that is, f(x)).

An important example concerns protection from blunt trauma by some form of cushioning or padding. Low stiffness is desirable at low energies of impact, but high stiffness is desirable at high energies: the reason is that if stiffness is low, “bottoming out” may occur, and the human will in effect be struck by the very stiff structure that the cushioning is intended to protect against. Thus a design that uses low stiffness material may be better for low energies of impact and a design that uses high stiffness material may be better for high energies of impact.

Some calculations will now be given, both to show the mechanics of calculating results using Equation (1), and to demonstrate the point just made of there being a danger if cushioning is too soft. In these calculations, a high score will reflect a bad outcome, as for incapacitation or HIC.

• Table 2 gives two possible examples of functions h(x), that are referred to as h_A and h_B. Design A performs better than B when x is Low and Medium, but performs worse when x is High. This might be the case if x is impact speed and design A refers to cushioning that is not stiff enough to prevent bottoming out when x is High.
• Table 3 is an example of a function p(h), showing what levels of p correspond to what levels of h.
• Table 4 gives two possible examples of functions f(x), that are referred to as f₁ and f₂. In the first case, Low x is most common, occurring with probability 0.5 (that is, on 50% of occasions). In the second case, High x is most common, occurring with probability 0.8 (that is, on 80% of occasions).

If the first set of probabilities represents the real world, design A is better, as Avp is 27.5 whereas Avp is 42.5 for design B (Table 5). However, if the second set of probabilities represents the real world, design B is better, as Avp is 67.5 whereas Avp is 82.5 for design A (Table 5). Expressed alternatively, set 2 gives a higher result than set 1 for both designs, but the increase is so much greater for design A than for design B that the ordering of these designs is reversed.

It is readily appreciated that this occurs because of the cross-over in Table 2 from h_B being greater than h_A (when x is Low or Medium) to h_B being less than h_A (when x is High). Bottoming out might cause this, but other reasons are also possible. For example, x might refer to different parts of the body, with design A giving better protection than B to the parts referred to as Low and Medium, but worse protection than B to the parts referred to as High.

If the different h's are valued according to the p's in Table 3, the h_A's from Table 1 will give rise to p's taken from the set {0, 25, 100}, and the h_B's will give rise to p's taken from the set {25, 50, 75}. Whether the average of numbers taken from {0, 25, 100} is larger or smaller than the average of numbers taken from {25, 50, 100} will obviously depend on the proportions in which they are taken.

• It is plain that to make a decision between design A and design B, the outcomes need to be valued or costed—the step of converting the descriptions h into the numbers p is essential—and averages calculated.
• It is probably desirable that the p's are dollar values, as then it will be possible to compare the difference between the Avp of alternative designs with the difference between the cost of the alternatives. For example, a more expensive design might perform as well as design A for Low and Medium x, and as well as design B for High x.

It will usually be appropriate to isolate the comparison of h_A and h_B, by using the same f in calculating the two means. But if there is a linkage between h_A and one set of real-world proportions f_A and between h_B and another set of real-world proportions f_B, one would average p(h_A) using f_A and average p(h_B) using f_B. The result will conflate the consequences of a change from h_A to h_B and a change from f_A to f_B.

Discussion

In some circumstances, it may be likely that h depends on something quantifiable, as well as on x. An example might be y = thickness of protection. With h now being h(x; y), Equation (1) provides a method of calculating the average cost or utility or incapacitation as a function of y, Avp(y).

When x is a measurement, it will often be the case that h increases as x increases, and p increases as h increases. For example, the greater speed is, the greater HIC is, and the greater the HIC, the worse will be the effects. However, riot control technologies may be an exception: utility as a function of energy may first increase (more energy = more effective), but then decrease (highest energy = too much risk of death or serious injury). It will be important to ensure this is correctly reflected in the function p(h), but it creates no problem for Equation (1); indeed, it will tend to reinforce the usefulness of summarising the trade-off in this way.

The examples discussed are concerned with tests of impacts of either penetrating or blunt objects with humans. Tests of impacts of objects with vehicles or structures, and tests of blast effects on humans, vehicles, and structures are not very different in principle. What of a completely different type of testing, that of people or teams of people, might this method—generalise, value, and average—be useful there too? Perhaps so: the effect of environmental conditions (for example) on task performance may be different for different people, particularly as a number of different factors may contribute to success (for example, specific ability, carefulness, creativity, persistence). Consequently, great care is needed in extrapolating from the test conditions to the real world. There is discussion of this in [11].

The aim—of making fuller use of a test result—is potentially widely useful. The method—generalise, value, average—is simple in concept, and quite general The deterrent to using or even proposing such a method is that it demands a lot of data that may be difficult to obtain. Pedestrian headform impacts test how good a cushion a car's front is. Cushioning has many other applications, and the subject draws on many disciplines. It seems possible, then, that something similar to Equation (1) has been developed previously, particularly since there is a history in road safety research of dissatisfaction with the limited conditions of the tests and a desire to draw wider inferences [3]. However, the present writer is not aware of any such development, either in road safety or any other field of testing. The exception is the procedure in [1].

Equation (1) highlights the use of generalise, value, average by Kokinakis and Sperrazza, and it is hoped that their procedure is thereby clarified. Secondly, the intention has been to demonstrate the common features of all testing protocols in which one set of conditions has to stand in for a great variety of real-world sets of conditions.

Acknowledgements

The Centre for Automotive Safety Research is supported by both the South Australian Department of Planning, Transport and Infrastructure and the South Australian Motor Accident Commission. The views expressed in this report are those of the author and do not necessarily represent those of the University of Adelaide or the funding organisations.

References

Author

Dr Paul Hutchinson is a Senior Research Fellow in the Centre for Automotive Safety Research, University of Adelaide, South Australia 5005. E-mail paul@casr.adelaide.edu.au, phone +61 8 8313 5997, fax +61 8 8232 4995.