System Dynamics Applications / 6 / 6.4

6.4 VALIDATION—CONSIDERATIONS FOR THE DESIGN OF TESTING

When we validate system dynamics models we seek to determine the extent to which two criteria are satisfied (Forrester, 1961: 115-129), the model must:

• generate behaviour that does not differ significantly from the real system, and
• explain real world behaviour through the structure and equations which reflect the real causal relationships in the real-world system.

The importance of the second criterion is that any number of models can be constructed to mimic the real world—that is, they can reproduce a given set of behaviours without faithfully representing real-world causal structures (cause-and-effect relationships).

Wittenberg (1992: 22-23) explains:

… one’s validation strategy depends largely on model purpose … [for] real-world systems, whose purpose is to solve a particular problem or understand a particular mode of behaviour … model and reference mode development are largely independent activities. Thus, while it is true that without a mental model and a purpose one would not know what behaviour is significant, it is also true that one can observe and record that behaviour without any knowledge of its underlying causal mechanisms. Choosing which variables appear in the reference mode is surely model-dependent; the shape of the reference mode is not.

This leads us to focus our validation activities for real-world problems on two types of test:

• structural tests, and
• behavioural tests.

Validation testing, therefore, aims to identify cause-and-effect mechanisms and determine the extent to which the way we have represented them in our models is a sufficiently faithful representation of the real world to meet our needs. We must remain mindful of the fact that we cannot establish truth through system dynamics modelling. The best we can achieve is confidence that our models are necessary and sufficient representations of real-world cause-and-effect structures.