Analysis, Causality and Proof in Safety Investigations

Executive summary

The ultimate purpose of a safety investigation is to enhance safety, and it is not the purpose to apportion blame or liability. A safety investigation into an occurrence (accident or incident) can enhance safety by identifying safety issues and communicating these issues to relevant organisations. It can also enhance safety by providing information about the circumstances of the occurrence and the factors involved in the development of the occurrence to the transportation industry. 

The quality of a safety investigation’s analysis activities plays a critical role in determining whether the investigation is successful in enhancing safety. However, safety investigations require analysis of complex sets of data and situations where the available data can be vague, incomplete and misleading. Despite its importance, complexity, and reliance on investigators’ judgements, analysis has been a neglected area in terms of standards, guidance and training of investigators in most organisations that conduct safety investigations. 

To address this situation, the Australian Transport Safety Bureau (ATSB) developed a comprehensive investigation analysis framework. The framework consists of: 

• a defined process or workflow for conducting analysis activities; 
• standardised terminology and definitions; 
• an accident development model (termed the ATSB ‘investigation analysis model’); 
• and policies, guidelines, tools and training for investigators. 

As with all analysis approaches, some concerns have been raised regarding aspects of the ATSB framework, particularly regarding the standard of proof used to determine contribution to the development of an occurrence and the nature of the ATSB investigation analysis model. In terms of standard of proof, the ATSB framework defines a ‘contributing safety factor’ as a safety factor that, if it had not occurred or existed at the relevant time, then either the occurrence would probably not have occurred, adverse consequences associated with the occurrence would probably not have occurred or have been as serious, or another contributing safety factor would probably not have occurred or existed. The term ‘probably’ was defined as being equivalent to ‘likely’ and meaning more than 66 per cent likelihood (a definition used by the Intergovernmental Panel on Climate Change). 

Because of its focus on future safety, the ATSB definition adopts a ‘link-by-link’ approach, where the judgement about whether a safety factor contributed to the development of an occurrence is made in terms of its relationship to another contributing safety factor. In contrast, other types of investigations (particularly those whose purpose is to determine responsibility) generally use a ‘relative-to occurrence’ approach. With the relative-to-occurrence approach, judgements of contribution are made in terms of the safety factor’s relationship to the occurrence itself. The ATSB analysis framework will involve a higher standard of proof than in Australian coronial inquests or civil legal proceedings for factors relatively close in proximity to the occurrence (that is, more than 66 per cent versus more than 50 per cent). But as an ATSB safety investigation proceeds to identify contributing safety factors more remote from the occurrence, the degree of relationship of the factors to the occurrence itself will generally decrease using the ATSB framework. 

Associated with the concept of standard of proof is the concept of standard of evidence, or the quantity or quality of evidence required before a decision maker can be satisfied that the relevant standard of proof has been met. In the Australian legal system, the ‘Briginshaw scale’ is used when making judgements about the standard of evidence. The scale involves considering the seriousness of a finding, the inherent unlikelihood of a finding, and the gravity of consequences that flow from a finding for the party or parties involved. The Briginshaw scale is not routinely incorporated into safety investigation methods. The scale is used to some extent in ATSB safety investigations, but there are several reasons to consider that it is not required or beneficial for the ATSB to apply the scale more broadly. 

The differences between the ATSB approach to determining contribution and other approaches may be a matter of nuance in many situations, and similar findings may result regardless of the approach being used. Nevertheless, there is also the potential for different sets of findings to be produced. More specifically, the ATSB’s link-by link approach together with a ‘probable’ standard of proof has the following advantages over many other investigation analysis approaches: 

• It better enables the search for potential safety issues, particularly those more remote from an occurrence. The enhanced searching will result in more safety issues being identified and communicated to relevant organisations to enhance safety. 
• It has greater potential for providing a richer or more detailed description of the factors involved in the development of an occurrence, which provides better learning opportunities for the transport industry.  
• It is more distinct from the approach used in legal proceedings for determining blame or liability. Therefore, there is less potential for the existence of barriers to learning or safety action due to an investigation’s findings being associated with such legal proceedings, or interpreted with such proceedings in mind. 

In terms of the ATSB investigation analysis model, it is based on the widely used Reason model of organisational accidents and consists of five levels of safety factors (occurrence events, individual actions, local conditions, risk controls and organisational influences). Concern has been raised that the model is biased towards finding problems at the higher levels of the model and that the pendulum has swung too far towards searching for organisational factors. The ATSB model does encourage investigators to look for problems with risk controls and organisational influences as, if there are problems in these areas, this is where significant safety enhancements can be made. However, the model is only used as one means to help identify potential safety factors. Before any findings are made about whether these potential factors contributed to the development of the occurrence, or were otherwise important, they need to be tested or verified. In the ATSB analysis framework, this involves using a structured process to examine the available evidence and conducting tests for existence, influence and importance. 

The ATSB believes that its investigation analysis framework is well suited to its role as an independent, no-blame safety investigation body. It is hoped and expected that ongoing development and provision of information about the framework can help the safety investigation field as a whole consider some important issues and help develop the best means of conducting safety investigations to enhance safety. Accordingly, any feedback or comment that any individual or organisation has regarding the ATSB analysis framework, ways to enhance the framework, ways for the ATSB to better communicate its findings, or any other matters discussed in this report would be gratefully received.