On 23 September 2018, during planned signal maintenance work, a wayside signalling system irregularity occurred where signal EJ45 at Eagle Junction, Queensland displayed an incorrect authority to an approaching train (DP41). The authority from signal EJ45 was invalid for the positioning of the 705 points ahead. Both the driver of train DP41 and a signal electrician noticed the irregularity, and reported it to the network control officer at the Mayne Control Centre. The network control officer directed the driver of the train to remain at the Eagle Junction platform. A short time later, another train (DG44) approached and crossed over the conflicting route. The actions of the driver of DP41, network control officer, and signal electrician identifying the irregularity and stopping the train mitigated the potential for a collision to occur.
What the ATSB found
The ATSB found that, following replacement of the 705C points machine, a fault was detected. The fault resulted in a mismatch between the position of the 705 points set, and the signal indication at the station (EJ45) and the Universal Train Control System display used by the network control officer. Despite not understanding the nature of the fault, for undetermined reasons, control of the 705 points was handed back to the network control officer. As such, train DP41’s route was set for the Airport Line, although the points were directed to the Doomben Line.
The new 705C points machine had been pre-wired in accordance with the Queensland Rail master circuit diagram. However, the diagram contained an error and did not reflect the wiring configuration of the old points machine, which had been functioning correctly until replacement. This error lead to signal EJ45 displaying an incorrect authority.
Queensland Rail did not have a process in place to check the master circuit diagram with the existing configuration of the in-field equipment before using the diagram for safety critical work.
What's been done as a result
Queensland Rail have introduced a procedure requiring correlation between the in-field signalling equipment and the master circuit diagrams before being used for safety critical work.
Safety message
The incorrect wiring configuration led to the signal interlocking system safeguards being circumvented. This incident highlights how accurate and up-to-date engineering documents that correlate with in‑field equipment are fundamental to the effectiveness of an engineered interlocked signalling system to maintain train separation.
The occurrence
Preparation for planned track work
Track work had been planned[1] between Eagle Junction (clear of the main and suburban lines) and Doomben, Queensland for Sunday 23 September 2018. This work included the replacement of the 705C (HW2000) points machine and timber bearers at Eagle Junction. At about 0627 Eastern Standard Time,[2] a protection officer (PO) contacted a Brisbane suburban network control officer (NCO) in the Mayne Control Centre[3] to arrange a track occupancy authority (TOA) for the work.[4] The work group consisted of a civil team, and a signalling team including a team leader and three signal electricians (E1, E2, E3).
At about 0633, the PO clipped and locked the 704 and 705A/B (Figure 1) points at Eagle Junction in the normal position.[5] For the 705A/B points, this meant that trains from the city were being directed towards the Airport Line and were prevented from entering the worksite at the 705C points on the Doomben Line. In addition, the 705C points were electrically isolated[6] from the signalling system to prevent the work affecting train operations over the 705A/B points on the suburban line. About an hour later, after the work group had completed a pre-work safety brief, the planned work commenced on 705C.
Figure 1: Overview of Eagle Junction showing the location of the points
Note: The suburban line to the airport is represented in orange, the suburban line from the airport is yellow and the Doomben Line is red.
Source: Google Earth, annotated by the ATSB
Replacement of the points machine
The work continued as planned with the removal of the old points machine and re-timbering underneath. The new pre-wired machine was installed then checked, mechanically and electrically, and found to be functioning correctly. A signal electrician (E1) then liaised with the PO to conduct further testing with the involvement of the NCO.
Testing of the points machine
The PO contacted the NCO to arrange access to the Eagle Junction area for the testing to occur. The PO unlocked and unclipped the 704 and 705A/B points, then confirmed the correct operation of 704 points with the NCO. The PO advised the NCO that the signal electricians would liaise directly with him to conduct the testing on the 705 points set.[7] Following this, a signal electrician removed the electrical isolation on the 705C points.
First test
At 1446, a signal electrician (E1) contacted the NCO to commence testing of the 705 point indications. The first test involved breaking detection[8] at the 705C point machine to validate the function of the vital interlocking system input. The test passed successfully with a loss of normal detection indicated to the NCO on the Universal Train Control (UTC) system interface (refer to section titled Signalling infrastructure).
Second test
The second test required the NCO to enter a command via the UTC system interface for the 705A/B and C points to motor to the reverse position, directing trains to the Doomben Line. The signal electrician located at the 705C points observed the point machine move into the reverse position. However, the NCO stated there was no corresponding detection indicated on the UTC display. That is, the display continued to show the 705 points were not detected.
The NCO asked if the 705A/B points were part of the work. The signal electrician confirmed with the NCO that 705A/B were not part of the scope of work, and requested more time for testing. The NCO stated the next train required the 705 points set to normal (directing trains from the city towards the Airport Line) and that further testing could continue after the train had passed. The NCO received approval from the signal electrician to command the 705 points set back to normal. The NCO confirmed the UTC displayed detection of the points in the normal position. The signal electrician then informed the NCO that 705C had not moved and were still orientated in the reverse position, out of correspondence with the UTC system indication.
The signal electrician interrupted the detection electrical circuit at the 705C point machine and it motored to the normal position, previously selected by the NCO. The UTC indicated a loss of detection as the 705C points drove from reverse to normal, and then the UTC again displayed detection of the points in the normal position.
The NCO asked the signal electrician if it was okay to use the suburban airport lines (Figure 1). The signal electrician confirmed it was now okay, but further testing would be required. Suburban passenger trains DP37 and DG40, to and from the airport, travelled through Eagle Junction without incident.
Additional testing and fault-finding
After the passage of trains DP37 and DG40, the signal electrician (E1) contacted the NCO to continue testing the 705C points. With each test the same issue existed, where the 705 points set were in reverse, but the UTC system indicated that they were set in the normal position.
The testing and fault-finding continued between three trains until about 1530 when the signalling team split up. The team leader and signal electrician (E2) proceeded to the Eagle Junction signal equipment room (SER) located 100 m west of the platform, and the signal electrician (E1) and assistant moved to the location boxes on the eastern end of the platform, near the 705A/B points. Another signal electrician (E3) remained at the 705C points location.
At about 1536, the team leader at the SER contacted the NCO. The team leader confirmed with the NCO the indicated position of the 705 points set on the UTC display and verified this against the display on the maintenance terminal in the SER. The other signal electrician (E1) located at the location boxes checked the position of the points and relayed information back to the team leader. After verifying the physical position of the 705A/B points with the signal electrician (E1), and confirming with the NCO that the 705 points set were in the normal position, the team leader realised that 705A/B were also out of correspondence with the indication displayed on the maintenance terminal. At this point, the NCO requested that the points be released from testing to allow routing of trains DP41 and DG44, to and from the airport respectively. The team leader handed control back to the NCO, who set the routes through Eagle Junction for DP41 and DG44.
Train DP41 approaching
At 1543:18, train DP41 approached Eagle Junction station from the city, heading towards the airport. At that time, the train’s forward facing video camera showed signal EJ45P (before the station) changing from a single yellow (proceed with caution) to green (proceed at authorised speed) (Figure 2).
Figure 2: Train DP41 approaching Eagle Junction station with signal EJ45P
Note: Signal EJ45P changing indication from yellowto green as DP41 approaches.
Source: Queensland Rail, annotated by the ATSB
Train DP41 arrived at the Eagle Junction platform for its scheduled stop, at which time the next signal ahead of the station, EJ45, was displaying a double-yellow indication to proceed with caution. At that instance, the driver saw workers at the end of the platform (including signal electrician E1 and the PO) and noticed that the orientation of the 705A/B points were in reverse directed towards Doomben, which conflicted with his route to the airport and the indication displayed by signal EJ45 (Figure 3).
Figure 3: Train DP41 at the platform with signal EJ45 and the 705 points incorrectly set
Note: Signal EJ45 indicating double-yellow. Route indicator not illuminated indicating points are set towards the Airport Line.Signal electrician E1 with the protection officer visible on the right. The gaps between the rails show that the points are in the reverse position.
Source: Queensland Rail, annotated by the ATSB
At around the same time, the signal electrician (E1) at the end of the platform noticed the irregularity between EJ45 and the 705 points set. The signal electrician visually indicated to the driver of train DP41 not to proceed. The driver contacted the NCO via radio to report the issue. The NCO directed the driver not to proceed until further instruction. The signal electrician also called for the detection to be broken at the 705C points. Train DP41 remained at the platform.
Train DG44 approaching
About 100 seconds after, train DG44 from the airport rounded the curve and approached Eagle Junction, across the conflicting route (Figure 4). The driver of DG44 overheard the conversation on the radio between the driver of DP41 and the NCO, and cautiously approached Eagle Junction platform for a scheduled stop.
As train DG44 was arriving at the platform, the signal electrician (E3) at the 705C points interrupted the detection electrical circuit, causing the 705 points set to motor to the normal position, as set by the NCO on the UTC system prior to the arrival of DP41 and DG44. As the points moved, signal EJ45 displayed a stop indication until the points were in the normal position and detected. Signal EJ45 then displayed a double yellow proceed indication, the correct indication for the configuration of the 705 points.
Figure 4: Train DG44 crossing the conflicting route
Note: Approaching train DG44 crossing the conflicting route.
Source: Queensland Rail, annotated by the ATSB
Train DP41 departing
As a result of the 705 points being reset, a ’restored in face of DP41’[9] alarm activated on the UTC system in the control centre. The NCO immediately called the signal electricians to ascertain the reason for the alarm. The team leader confirmed that no further work would be performed on the 705 points. Several minutes later, after confirming the correct orientation of the points with the team leader, the NCO permitted train DP41 to depart. Train DP41 departed without further incident.
Eagle Junction station is located about 7.5 km north of Brisbane Central station. The station services the Airport, Doomben, Shorncliffe, and Sunshine Coast Lines with regular passenger train services. Point set 705A/B and C were located on the down side of the Eagle Junction station and formed the junction between the Airport and Doomben Lines.
Signalling infrastructure
The wayside[10] signalling infrastructure at Eagle Junction was remotely operated by a network control officer (NCO) located in the Mayne Control Centre at Bowen Hills. The NCO used the Queensland Rail universal train control (UTC) system to set the routes required, and place blocks on sections of the track for maintenance and other purposes, enabling the safe passage of trains throughout the NCO’s assigned control area. The UTC showed a green line to represent the route set, a red line for an occupation/lock, and a yellow line when the section of track was not detected/error. Figure 5 shows a representation of what the UTC displayed when train DP41 was at the Eagle Junction platform.
Figure 5: UTC playback of the Eagle Junction area at 1545:05
Note:This figure is representative of Figure 4.
Source: Queensland Rail, annotated by the ATSB
The UTC is a non-vital[11] computer-based system that interfaces with the vital signal interlocking equipment in the field to control the point machines and signals. The signal interlocking equipment sets the aspect displayed on the signal for the selected route, subject to the correct conditions being met. The interlocking receives information from the wayside equipment and transmits information of the status of the various track circuits, point machine position and signals back to the UTC system for display to the NCO (Figure 6).
Figure 6: Interface between the UTC system, vital signal interlocking equipment and the field infrastructure
Source: ATSB
Signal indications
The Queensland Rail signalling system at Eagle Junction used four-aspect signalling to display proceed authorities to drivers. Table 1 illustrates the four possible aspect combinations and their meaning.
Table 1: Four aspect signalling system indications
Table 2: Route indications
The renewal of the 705C points formed part of the 2017 program of work for the south-east Queensland point machine upgrade to increase reliability, and remove asbestos. The replacement of the 705C points did not occur in 2017 and carried over to the 2018 work program.
In July 2018, track possession planning commenced for the renewal of the 705C points. The original possession planning called for overnight works, between the last and first train services through Eagle Junction. However, to better accommodate staffing availability and fatigue management, the plan was amended for the work to be undertaken during the day, in negotiation with the NCO between train services. This plan coincided with an availability in the signal team’s work plan.
The signal team who carried out the removal and installation of the 705C points machine were all employed by Queensland Rail and were appropriately qualified. The planned work performed on this day was described by the signal team as routine and had been previously performed at other locations without issue.
705 points set
History
Prior to 2001, Eagle Junction had the 705A and B points, which consisted of switch-blade components with a fixed crossing. With the commissioning of the Airport Line in 2001, the points were re-configured and upgraded to 705A/B and C points, and included a movable swing-nose crossing.
705C points machine master circuit diagram
The Queensland Rail master circuit diagram (#3225) for the 705C points machine was first issued in 1999. In-line with their procedures, any changes made affecting the points was to be reflected on the circuit diagram and approved by an engineer. At the time of the incident, version four of the circuit diagram was current, dated June 2014. Variations made since initial issue were annotated on the diagram, none of which mentioned changes specifically relating to the 705C points machine. However, Queensland Rail noted that their documents only had to be retained for a period of 10 years, which meant that the reasoning behind any changes made beyond this could not be traced.
The master circuit diagram was used by the team leader to pre-wire the new 705C points machine prior to the day of installation. Figure 7 shows the way in which the new point machine was wired (in red).
On the day of the incident, the work group removed the old machine and installed the new pre‑wired machine. During this process, there was no requirement to verify the wiring configuration of the old machine with the current master circuit diagram prior to its removal.
On-site fault finding
The work group continued fault-finding in an attempt to identify the reason for the irregularity between the signalling system and the points. Firstly, the wiring of the new 705C points machine was checked and verified with the master circuit diagram (#3225). The fault-finding was then expanded to include the circuit diagram for the 705A/B points machine and full master circuit diagrams at the SER. After reviewing these diagrams, the signal electricians determined that there was an error with the master circuit diagram for the 705C points machine.
The signal interlocking system relies on polarity signals from in-field equipment to determine the binary condition of equipment. For example, a positive voltage would register in the signal interlocking system as a normal points position, and a negative voltage indicated a reverse points position. This information is then utilised by the system to logically determine if a proposed movement is allowed or not. However, Queensland Rail reported that the master circuit diagram for the 705C points machine was drawn in a way in which the ‘configuration essentially sets the system to understand the points are in normal whether the points are in normal or reverse’. They also concluded that, although the diagram was incorrect, the removed points machine had been correctly configured and had worked as expected without fault.
Figure 8 (left) represents what the in-field wiring (in red) would have looked like before the 705C point machine was removed. This circuit also represents the change that was later approved following the incident. For comparison, the incorrect wiring configuration from the master circuit diagram is show on the right.
Figure 8: Comparison of the in-field equipment wiring (left) with the 705C points machine circuit diagram (right)
Source: Queensland Rail, annotated by the ATSB
After identifying the error with the diagram, all further work was suspended. The NCO arranged for 705A/B points to be clipped and locked, and electrically blocked, until an independent signalling engineer could approve the necessary changes to the master circuit diagram, which occurred later that night.
Queensland Rail indicated that, the configurations shown above (Figure 8) were both correct ways for wiring a points machine. However, only one of these was applicable to the set-up at Eagle Junction. Hence, the error in the master wiring diagram may not have been obvious to the signal electricians by visual inspection.
From the evidence available, the following findings are made with respect to the signalling irregularity at Eagle Junction, Queensland, on 23 September 2018. These findings should not be read as apportioning blame or liability to any particular organisation or individual.
Safety issues, or system problems, are highlighted in bold to emphasise their importance. A safety issue is an event or condition that increases safety risk and (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.
Contributing factors
Even though an abnormal condition, which was not understood, was identified during testing of the signalling system, the train was authorised to proceed using the system. This resulted in train DP41 approaching a junction where there was a mismatch between the signal indication and the position of the points.
The Queensland Rail master circuit diagram did not reflect the existing configuration of the removed 705C point machine. This resulted in the new pre-wired machine being installed with a wiring error, which led to the abnormal condition and signalling system irregularity.
Queensland Rail did not have a procedure in place to cross-check a master circuit diagram with the existing configuration of the in-field equipment before using the diagram for safety critical work. This removed an opportunity to detect any error in master circuit diagrams. [Safety issue]
Safety analysis
Introduction
During planned signal maintenance work, a wayside signalling system irregularity occurred where signal EJ45 at Eagle Junction, Queensland displayed an incorrect authority to an approaching train (DP41). The authority from signal EJ45 was invalid for the positioning of the 705 points ahead. Both the driver of train DP41 and a signal electrician noticed the irregularity and the train was prevented from proceeding by the network control officer (NCO). A short time later, another train approached and crossed over the conflicting route. The actions of the driver of DP41, NCO, and signal electrician identifying the irregularity and stopping the train mitigated the potential for a collision to occur.
This analysis will examine the signal irregularity, how the pre-wired 705C points machine affected the signalling system, and why the error in the master circuit diagram for the machine was not detected before installation.
Signal irregularity
Following replacement of the 705C points machine, the signal electricians tested the functionality of the system and detected a fault. The fault resulted in a mismatch between the position of the 705 points set, and the signal indication at the station and the Universal Train Control System. While attempting to locate the fault, trains continued to operate normally over the points without issue. This occurred due to the signal electrician breaking detection at 705C, following each test but before the next train, eliminating the mismatch. However, this did not occur before train DP41 approached the station as the signal electrician had moved away from 705C to fault-find.
Despite not understanding the nature of the fault at that time, for undetermined reasons, control of the 705 points was handed back to the NCO. However, it was possible that the team leader expected the mismatch to self-correct, consistent with previous experiences with trains over that section. As such, the NCO set DP41’s route for the Airport Line, although the points were directed to the Doomben Line.
Points machine pre-wired
Queensland Rail required the master circuit diagram to be updated and approved when changes were made to the signalling equipment. However, post-incident fault-finding established that the diagram did not reflect the existing wiring configuration of the removed 705C points machine.
While minor changes over the years were noted on the diagram, there was no reference to the wiring configuration having been altered. Therefore, as the removed machine had been functioning correctly until replacement, it was likely that, when the points were upgraded to accommodate the Airport Line, changes were made to the machine but the master circuit diagram was not updated to reflect these modifications. Since that time, maintenance had been routinely performed on the machine, however, it was likely that this did not involve comparing the in-field wiring with the master circuit diagram.
Consequently, the new 705C points machine was wired in accordance with the master circuit diagram, and a wiring error was inadvertently introduced. This resulted in the Universal Train Control and signal interlocking systems indicating the 705 points set were in the normal position, irrespective of whether they were physically in normal or reverse.
Procedures
When replacing the 705C points machine, the existing machine was removed and the new, pre‑wired machine installed. During this process, there was no requirement to cross-check the wiring configuration of the existing machine with the master circuit diagram prior to removal. Without this check, this placed sole reliance on the diagram being updated each time changes were made to the points machine. This removed an opportunity to detect any errors between the diagram and the in-field equipment.
Safety issues and actions
The safety issues identified during this investigation are listed in the Findings and Safety issues and actions sections of this report. The Australian Transport Safety Bureau (ATSB) expects that all safety issues identified by the investigation should be addressed by the relevant organisation(s). In addressing those issues, the ATSB prefers to encourage relevant organisation(s) to proactively initiate safety action, rather than to issue formal safety recommendations or safety advisory notices.
Depending on the level of risk of the safety issue, the extent of corrective action taken by the relevant organisation, or the desirability of directing a broad safety message to the [aviation, marine, rail - as applicable] industry, the ATSB may issue safety recommendations or safety advisory notices as part of the final report.
All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.
Descriptions of each safety issue, and any associated safety recommendations, are detailed below. Click the link to read the full safety issue description, including the issue status and any safety action/s taken. Safety issues and actions are updated on this website when safety issue owners provide further information concerning the implementation of safety action.
No procedure to cross-check master circuit diagram
Safety issue description: Queensland Rail did not have a procedure in place to cross-check a master circuit diagram with the existing configuration of the in-field equipment before using the diagram for safety critical work. This removed an opportunity to detect any error in master circuit diagrams.
Sources and submissions
Sources of information
The sources of information during the investigation included the:
Queensland Rail
Drivers of DP41 and DG44
Network control officer
Signalling team.
Submissions
Under Part 4, Division 2 (Investigation Reports), Section 26 of the Transport Safety Investigation Act 2003 (the Act), the Australian Transport Safety Bureau (ATSB) may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. Section 26 (1) (a) of the Act allows a person receiving a draft report to make submissions to the ATSB about the draft report.
A draft of this report was provided to Queensland Rail, the Office of the National Rail Safety Regulator, and directly involved individuals.
Submissions were received from Queensland Rail and the Office of the National Rail Safety Regulator. The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
On 8 September 2018, a rail safety worker (RSW) was nearly struck by two trains within the airport tunnel, near Redfern, New South Wales. The RSW left their worksite in the Eastern Suburbs Railway (ESR) tunnel at Platform 11, Redfern, to find a toilet. In attempting to find a toilet, the RSW entered an adjacent worksite on the concourse level at Redfern Station. The RSW was then directed by a worker through the worksite and on to Lawson Street, Redfern. The RSW utilised the public toilets at the station but, was prevented from returning to his worksite due to the concourse level worksite. In attempting to return to the worksite, the RSW entered the danger zone at Platform 10, Redfern, before walking towards Central Station. On arriving at the entrance to the airport tunnel, the RSW mistakenly entered the tunnel in the belief he could return to his worksite.
While in the tunnel, the RSW reported hearing a train approaching, moved clear of the track and clung to the wall of the tunnel just before train 89-K passed his location. Train 89-K came to an emergency stop. After the near miss with 89-K, the RSW quickly exited the tunnel and crossed in the path of a second train 88-C near the mouth of the Up side of the tunnel. Following the near misses, the RSW returned to Redfern Station via the same path and exited the station by the stairs at the Erskineville end of Platform 10. The RSW located the maintenance entrance to the Eastern Suburbs Railway tunnel and completed this shift without reporting the incident until the afternoon.
What the ATSB found
The ATSB found that the RSW had not been briefed on the concourse level worksite or the requirement to access Redfern Station from Marian Street before commencing work. The RSW entered Redfern Station from the Eastern Suburbs Railway tunnel without receiving or seeking a project site induction and left the protected area to find a toilet. The RSW attempted to return to his worksite believing he could locate the entrance from the city side of Redfern, and entered the airport tunnel, which was live at the time.
Following the near misses with 89-K and 88-C, the RSW returned to Redfern and entered the worksite undetected after locating the entrance off Marian Street. The RSW signed the project site induction record sheet, indicating he received an induction when he was not present.
What's been done as a result
The rail safety worker was removed from safe working duties by Sydney Trains following this incident and placed on restricted duties.
Sydney Trains advised that they acknowledge the safety issue and will follow their business processes to address the safety issue. Additionally, they advised a number of briefings were conducted following the occurrence to communicate lessons learned with the workgroup.
Safety message
This incident highlights the importance of planning and controlling worksites within the rail corridor, in particular, when working within tunnels and at platforms where access can be from multiple locations.
Workers must ensure that they receive the required safety briefing prior to undertaking work or entering a new work area, and seek further information if required prior to starting work. Workers are also advised to stop and find a safe place in the event they become lost or separated from their work group. Workers should also report incidents in a timely manner.
The occurrence
What happened
On the morning of 8 September 2018, a project (Job No.336) to partially replace the railway sleepers within the Eastern Suburbs Railway (ESR) was being undertaken between 1.150 km[1] and 1.450 km at Redfern Platform 11. Assisting with this project were two groups of Signal Electricians providing support to the work group and protecting signalling equipment during the work.
At 0800,[2] a group of signal electricians consisting of a Signal Electrician (RSW)[3] and Team Leader (T/L) received a worksite protection pre-work briefing from the Protection Officer (PO) for the project. The brief was conducted near the hi-rail take off pad[4] located at 2.867 km near Erskineville. The section of track was within the protection of a Local Possession Authority (LPA)[5] between Bondi Junction and Hurstville.
Following the briefing, the RSW and T/L walked along the Illawarra Relief Up[6] within the ESR tunnel, suppressing train stops and checking signalling equipment along the way, before arriving at Redfern Platform 11. At 0840, the RSW left the worksite at Platform 11 and entered an adjacent worksite on the concourse level to find a toilet. This adjacent worksite was part of the Redfern Station entry upgrade at Lawson Street on the concourse level. A worker in the Redfern Station worksite directed the RSW through the area and on to Lawson Street. The RSW re-entered Redfern Station and utilised the public toilets at the station until 0856.
The RSW was unable to return to Platform 11 via the same path taken, due to the concourse level worksite. At approximately 0857, the RSW entered the danger zone at the city end of Redfern Platform 10 at 1.213 km. The RSW walked in an Up direction, attempting to locate the portal[7] of the ESR tunnel to return to the worksite. The RSW walked approximately 550 m along live track and access roads, before entering what they believed was the ESR portal. In fact, the RSW had entered the portal of the Airport Line (APL) tunnel at 0.626 km. The RSW walked in a Down direction in the tunnel believing that they could return to their worksite.
At 0906, train 89-K consisting of an eight-car Waratah departed Central, bound for Macarthur via the APL. Shortly after entering the APL tunnel, the driver of 89-K saw a person in high-visibility clothing in the Down four-foot[8] approximately 120 m inside the tunnel. The front-of-train closed circuit television (CCTV) footage from 89-K showed the RSW jumping clear of the track and clinging to the cable troughing on the tunnel wall, three seconds before the train passed at approximately 45 km/h. The driver of 89-K made an emergency brake application and the train came to a complete stop at 0907. The driver then reported a near miss to the area controller.
Following the near miss with 89-K, the RSW ran in an Up direction, between the wall of the tunnel and stationary train. On arriving at the rear car of 89-K, the RSW crossed the Down track and on to the Up track of the APL and in front of the approaching Up service 88-C. The driver of 88-C made an emergency brake application and halted the train at 0908. The driver of 88-C also reported a near miss to the area controller.
The area controller began warning trains in the area of a possible trespasser as the identity of the person was unknown. Warnings to trains of trespassers ceased at 0933 after no further sightings were made.
At 0915, the RSW returned to Redfern Platform 10 and exited the country end of the platform via the stairs to Marian Street. The RSW then located an entrance to access the ESR through an old maintenance depot located off Marian Street.
The RSW completed the remainder of his rostered shift at 1500 without advising of the near misses. Following this, the RSW sought medical treatment for a hand injury. The RSW reportedly cut his hand when he slipped and fell in the four-foot, before jumping clear of train 89-K and onto the cable troughing.
At 1645, the RSW reported the incident to the Shift Manager (SM) at the Rail Management Centre (RMC). Post-incident drug and alcohol testing was not possible due to the late reporting of the incident. Additionally, the RSW was at home at the time of reporting.
There are a number of terms used within the rail industry to describe the different areas within the rail corridor (Figure 1), these are:[9]
Danger zone, any area within 3 m horizontally and either above or below this area, unless constantly in a safe place.
Safe place, an area where a worker or their equipment cannot be struck by rail traffic. A safe place can be created within the danger zone, through the use of recognised protection arrangements.
Four-foot (4 ft), the spacing between the two railway tracks is referred to as the 4ft based on the railway gauge of 4 ft 8.5” (1435mm).
Six-foot (6 ft), the spacing between two sets of tracks.
Cess, this is the area between the outer most rail and the fence line or 15 m in non-fenced areas.
Figure 1: Rail corridor diagram
The figure shows two workers within the danger zone using recognised protection arrangements to create a safe place.
Source: Sydney Trains
Incident locations
Redfern Station
Redfern Station is located at 1.204 km and consists of twelve platforms, which service both Sydney Trains and NSW Trains services. Platforms 11 and 12 are the only underground platforms at Redfern, with these platforms forming part of the ESR.
At the time of the occurrence, Platforms 11 and 12 were closed to the public due to the partial re-sleepering project at Platform 11. Additionally, a project to redesign the station entrance on the corner of Gibbons and Lawson Street was underway. Barriers were erected near the top of the stairs and escalators to prevent access to this worksite from the ESR.
Access to Platform 11 and 12 was via a disused maintenance depot off Marian Street, Redfern. To access the toilets from the ESR, a worker needed to exit Platform 11 using the stairs and enter the office area of the disused maintenance depot. These toilets were the designated toilets for both worksites at Redfern.
Eastern Suburbs Railway
The construction of the ESR was completed in 1979, with the railway running between Erskineville and Bondi Junction. This railway primarily consists of underground tunnels with some short sections of viaducts. There were a number of construction methods used during the construction the ESR tunnel, resulting in different tunnel profiles along the length of the tunnel.[10] Between Erskineville and Redfern, the tunnel was constructed using cut and cover technique,[11] giving this section a square shape. The remainder of the tunnel is a horseshoe arch-shaped and formed with drilling and blasting techniques.
The ESR tunnel begins north of Erskineville with the portal located at 2.634 km (Figure 2). At the entrance the Up and Down Illawarra Relief (IR) are located side by side with a dividing wall separating the two lines. There are refuges located along the length of the tunnel to provide workers with a safe place if required.
Redfern is the first station with Platforms 11 and 12 beginning at 1.409 km in the Up direction.
Airport Line
The construction of the APL was completed in 2000, with the railway running between Central and Wolli Creek. The APL runs within a dual-line tunnel, between the portal entrance near Prince Alfred Park at 0.626 km (Figure 2) and Wolli Creek at 9.358 km. The tunnel was constructed using tunnel boring equipment producing a round tunnel. The Sydney Trains’ Worksite Protection Hazardous Locations Register (WPHLR) listed the APL tunnel as not having dedicated safe places available in the tunnel. At the time of the occurrence, there was no worksite protection in place for the APL. Rail traffic could operate at the maximum track speed of 80 km/h through the tunnel.
Figure 2: Eastern Suburbs Railway and Airport Line Tunnels
This map shows the underground path of the Eastern Suburbs Railway tunnel in orange and Airport Line tunnel in white.
Source: Google Earth, annotated by OTSI
Planning and protecting worksites in the rail corridor
Prior to conducting work within the rail corridor, the safety risks are to be assessed to determine the level of safeworking protection required. This is set out in Network Rule NWT 300 Planning work in the Rail Corridor. The preferred methods for worksite protection when working on track are Local Possession Authority or Track Occupancy Authorities (TOA).
Included within NWT300 was a requirement to conduct a safety assessment including safe access to and from the worksite. The PO must also brief workers about the worksite protection and safety controls prior to workers commencing work.
The concourse level worksite was within the rail corridor but did not encroach on the danger zone. Therefore, this worksite did not need assessment in accordance with NWT300. Barriers were erected around the tiling worksite to prevent unauthorised access to the area.
Local possession authority
Network Rule NWT 302 Local Possession Authority details the rules for authorising, issuing and using an LPA. An LPA allows for the closure of a section/s of track for a period of time allowing for a single worksite or multiple worksites within the limits of an LPA.
Network Procedure NPR 700 Using a Local Possession Authority, defines the process for implementing and protecting an LPA. Red flags and red lights are used as a visual means of delineating the protected area of the LPA. The delineation flags and lights are to be placed in the four-foot.
Track possession planning
Planning of track possessions is completed in accordance with Sydney Trains’ process and procedures. Prior to conducting track possessions, pre-possession meetings are held in accordance with Assets Scheduling & Delivery Coordination Pre-Possession Meeting Procedure(AS&DC Pre-Possession Meeting Procedure) and Network Access Manual Volume 2 – Resource Scheduling & Coordination in Possessions (NAMV2). These procedures detail the required attendance at the pre-possession meeting and the responsibility for cascading this information to staff at their pre-work briefings. These procedures also detail the necessary information to be presented and confirmed during the pre-possession meeting, including:
Possession limits and interfaces.
Any interfaces with other possessions or other project works (including external companies and authorities).
Hazards as identified through the risk mitigation plan (adjacent worksites).
The AS&DC Pre-Possession Meeting Procedure additionally specifies that:
Sydney Trains or contractor company representatives at the Prepossession Meeting (e.g. Construction Managers, Worksite Supervisors, Safe Working service providers) who are responsible for inducting workers after the Pre-possession Meeting must:
Not permit any person to work on their site/project unless that person has received an appropriate level of induction and a site-specific Pre-Work Briefing.
Possession briefing notes are developed through the planning phase to provide details of the track possession for distribution. The possession briefing notes relevant to this project[12] specified the following in relation to possession protocols and responsibilities:
A qualified and accountable representative from each worksite must attend the pre possession meeting and make sure that all relevant details are correct and included in the specific worksite briefing.
It is the responsibility of persons representing worksites to cascade the pre possession briefing details to other staff who will carry out safeworking duties during the possession for the jobs that they are representing.
Project site induction briefing
Workers were required to undertake a specific project site induction briefing prior to conducting work within the worksite at Redfern. This project site induction provided the project scope and detailed the hazards and risk controls for the project, and was separate to the worksite protection brief for the LPA. The project site induction was delivered on the mezzanine level at Redfern; this level is only accessible via the stairs from the platform or via the disused maintenance depot (Figure 3 and Figure 4). All workers entering the worksite from Marian Street walked past this location before accessing the platform level.
The location of the amenities were listed within the brief as ‘Toilets are located at the station’, but provided no details on how to access this location.
Figure 3: Mezzanine level and briefing area
Source: OTSI
Rail safety worker
The RSW was performing the role of a Signal Electrician on the day of the occurrence. This individual had approximately 13 years’ experience working within the rail industry. They held the required qualifications for working on signalling equipment and working within the rail corridor.
The RSW additionally held the qualification of a Protection Officer Level 1 (PO1), this qualification allowed the worker to perform the following tasks:[13]
assess risk associated with work within the rail corridor
identify the level of protection required
implement lookout working and ASB methods of worksite protection, and
act as a lookout in the rail corridor.
Path taken by the rail safety worker
The RSW received a worksite protection brief near Erskineville, before walking to Redfern Platform 11 within the ESR. On arriving at the worksite at Redfern, the RSW left the platform to find a toilet on the concourse exiting the worksite at Redfern via the escalator (Figure 4).
Figure 4: Path taken by the RSW between Erskineville and Redfern
This diagram depicts a side sectional view allowing areas both above and below ground to be seen. The red dotted line depicts the path of the RSW.
Source: OTSI
After using the public toilet at Redfern Station, the RSW entered the danger zone at the city side of Platform 10 (Figure 5) before entering the APL tunnel.
Following the near misses, the RSW returned to Platform 10, exiting via the stairs before locating the entrance off Marian Street and returning to the worksite (Figure 6).
Figure 5: Path taken by the RSW from Redfern to entering the Airport Tunnel
Source: Google Maps and OTSI, annotated by OTSI
Figure 6: Path taken by the RSW from the Airport Tunnel to return to the ESR worksite
The RSW received a worksite protection pre-work brief at 0800, near the hi-rail take off pad at Erskineville. The briefing was delivered by the PO for Job No. 336. This PO held the qualification as a PO2.[14] The worksite protection plan clearly detailed the limits of the LPA and areas of live track were shown on a map.
The worksite protection pre-work briefing listed the participant acknowledgements (Figure 7). In signing this briefing, workers acknowledge that the items shown within the briefing are accepted and have been explained.
At the time the RSW and T/L were briefed at Erskineville, there were three other workers briefed. These workers signed this worksite protection pre-work briefing, but there was no record of these workers signing the project site induction as required for entering the worksite. It was not able to be determined if these workers were advised of the requirement to undertake a project site induction on arrival to Redfern as part of this brief.
A review of the worksite protection pre-work briefing and project site induction briefing showed that between 0200 and 0925 hours, a total of 30 workers signed on to the worksite protection pre-work briefing. Of these 30 workers, only 21 workers signed the project site induction for the worksite at Redfern.
The participant acknowledgement also stated:
have been briefed on the hazards of adjoining worksites/processes
At interview, the RSW reported not being advised of the concourse level worksite and stated that he was unaware of the location of the toilets.
The hazards and controls for the worksite between 1.150 km and 1.450 km were included in the briefing (Figure 8). The hazards and controls assessed did not list the adjacent worksite on the concourse level or the control of entering via Marian Street, due to the concourse level worksite. The RSW received the worksite protection pre-work briefing almost 1.5 km away from Redfern. If the RSW was advised, it is not clear how effectively this information could be conveyed to a worker entering via the tunnel.
The concourse level worksite in itself was not within the rail corridor and would not normally be considered an adjacent worksite. This worksite did, however, affect the potential access and egress to the ESR and should have been considered.
Figure 8: Worksite protection pre-work briefing – Hazards and Controls
Source: Sydney Trains, redacted by OTSI
The location of the briefing was delivered within the limits of the LPA, however there was no specific details for the RSW and T/L conducting work between the hi-rail take off pad at 2.867 km and Redfern Station.
The project site induction briefing also lists an acknowledgement section similar to that in the worksite protection pre-work briefing (Figure 9). Of note, both briefings list ‘have been briefed on the contents of the Worksite Protection Plan’. When the two briefing attendance sheets are compared, there are a large number of workers who signed the briefing almost two hours prior to receiving the worksite protection pre-work briefing. This may present a situation where workers commence work without receiving all required briefings, specifically the Worksite Protection Plan.
Figure 9: Project Site Induction Briefing Participant Acknowledgement
Source: Sydney Trains
Review of the PO’s diary notes showed a number of briefings being completed with a separate task then being completed shortly afterwards. The short space of time between the PO delivering a briefing and the start of a subsequent task carried out by the PO probably reduced the effectiveness of the briefings.
At 0410, the PO for Job No.336 received a briefing with the PO coordinating the combined worksite Zone B (the LPA covered multiple areas and this section was within Zone B of the possession). The diary notes for the PO coordinating the combined worksite noted this briefing contained information for ‘all hazards and controls to be implemented and CWS (combined worksite coordinator) B protection details’.
At 0412, the PO for Job No.336 replaced delineation flags and red lights between 1.190 km and 1.430 km on the Up Illawarra relief line.
At 0415, the PO for Job No.336 provided a worksite protection pre-work brief to 17 workers.
At 0416, the PO notes ‘Work started on Up Illawarra relief Platform 11 Redfern Station’.
Access and egress
The concourse level worksite at Redfern Station required workers accessing the ESR to enter the station via Marian Street. On entering the disused maintenance depot, facilities including a meal room, and toilets were available for the workers associated with the project works.
A briefing area had been set up on the mezzanine level opposite the door leading from the maintenance depot in to the station mezzanine level (Figure 3). The location of this briefing area suggests consideration was given to ensure that all workers entering from Marian Street would receive a project site induction.
While there was consideration for workers entering via Marian Street, there did not appear to have been consideration given to workers entering the worksite from the ESR. The RSW and T/L entered the project worksite from the ESR without receiving a project site induction or being detected as such.
The briefing area on the mezzanine level was not continually monitored to ensure that all new workers entering via from Marian Street were detected and briefed. When the RSW returned to Redfern after 0915, they were able to enter the worksite undetected. The RSW then signed the project site induction briefing attendance sheet as receiving a brief at 0900.
The T/L did not sign the project site induction attendance sheet, however it could not be confirmed if the T/L did receive a brief while the RSW was absent or not. In either case, at least two workers entered the project site undetected and without receiving a project site induction.
The barricades erected for the concourse level worksite did not prevent the RSW from entering the adjacent worksite, allowing the worker to leave the protection of the LPA.
Pre-possession planning
Sydney Trains advised that it was the responsibility of the worksite supervisor for Job No.336 for cascading the information from the pre-possession planning meetings.
The project owner requested support from the Signals Support Manager (SSM) through a resource request sheet (Figure 10).This request was for signals staff to attend site visits and a whiteboard meeting for the project.
Figure 10: Resource Request Sheet
Source: Sydney Trains, redacted by OTSI
Sydney Trains’ investigation reported that:
The signals support manager reported not receiving any documentation relating to job 336. Neither the signals support manager or team leader attended a site walk through completed one week prior to the possession.
Sydney Trains advised the ATSB that while safeworking is planned in accordance with their procedures and the project planning is carried out through other processes, it is not uncommon for safeworking issues to be raised at the project planning meetings. The project planning process incorporates whiteboard meetings to enable stakeholders to collect and share information.
At interview, it was reported that the whiteboard meetings are typically held prior to the commencement of planned work and all aspects of the project are discussed. From the evidence available, the whiteboard meeting for this project did occur, however the signal support team did not attended the meeting or site visits prior to the possession.
The AS&DC Pre-Possession Meeting Procedure and NAVM2 requires the pre-possession planning information to be communicated to all persons undertaking rail safety work. The whiteboard meeting was used as the method to conveying the pre-possession planning information to the relevant stakeholders. Sydney Trains do not have a process to hold a whiteboard meeting prior to commencing planned maintenance and to ensure that all workers received the required information.
Sydney Trains reported that staff working on Job No.336 were advised in the week prior of the requirement to access Platform 11 via Marian Street. The ATSB was told this advice was communicated verbally and in text, however no evidence of this was presented.
The Sydney Trains’ investigation stated that the T/L was advised of the requirement to access Platform 11 via Marian Street, during a phone call at 0509 on the morning of 8 September 2019. This was disputed by the T/L, who reported they did not receive this information until arriving at Platform 11 and after the RSW had left. If the T/L was advised of this information earlier, this was not then communicated to the RSW.
Decisions and actions of the rail safety worker
Following the briefing at Erskineville, the RSW walked with his T/L from this location through the ESR tunnel towards Redfern Platform 11. On arriving at Redfern, the RSW and T/L entered the project worksite without receiving or seeking a project site induction.
Sydney Trains reported that the RSW asked the T/L the location of the toilet, which was not known to either worker. Further, they stated the RSW had previously worked at Redfern, Platform 1 and reportedly knew the location of the public toilets on the concourse. At interview, the RSW advised the ATSB that he did not know the location of the toilets. It was not able to be determined if the RSW did or did not know the location of the toilets on the concourse. If the RSW was aware, this may have led the RSW to leave the platform via the escalator as the most direct route.
Leaving the platform via the escalator meant the RSW missed the location of the project site induction area and amenities on the mezzanine level. The RSW then pushed past barricades erected for the tiling worksite.
Review of the available CCTV footage shows the RSW enter the concourse level worksite, before being directed though the worksite on to Lawson Street. At interview, the RSW reported a person within the tiling worksite advised that they could not return via their worksite and advised that access to the ESR needed to be from the platform outside.
The RSW reported thinking that he could find a tunnel portal at the city end to return to his worksite. After using the public toilets located on the concourse at Redfern, the RSW walked towards Platform 10 and entered the danger zone at the city end of the platform. Around the time the RSW entered the danger zone at Redfern Platform 10, there were two other work groups in the area. The RSW reported seeing workers on the access road and possibly associated the workers as being involved with the same project, but did not ask these workers for assistance. The access road is not visible from the end of Platform 10 and would only be visible to the RSW after already entering the danger zone. The RSW continued walking before mistakenly entering the portal of the APL tunnel and not the ESR as he thought.
The RSW had worked across different parts of the rail network in approximately 13 years of working with Sydney Trains, including Platform 1, Redfern. Once the RSW was outside the protection of the LPA, he needed to make cognitive decisions in an effort to return to his worksite. While the RSW was qualified and experienced, in this instance, the RSW did not identify he was not where he thought he was. It is possible that the RSW’s plan of returning to his worksite, lead to cognitive fixation, also referred to as plan continuation. Plan continuation refers to the idea, that an original plan is developed, which initially appears correct, with the person being locked into their plan. As the plan progresses, the individual may overlook or disregard cues indicating their plan should be altered or abandoned (Dekker, 2011).
The RSW reported that he had not worked in the ESR between Erskineville and Redfern prior to this occurrence, but had worked in the ESR between Woolloomooloo and Bondi Junction. The RSW stated he had never worked in the APL. The shape of the ESR tunnel varies along the length of the tunnel as a result of different construction methods used throughout the tunnel. It is possible that the RSW did not identify the differences between the two tunnels when entering the APL tunnel. This may be a result of the worker’s fixation in attempting to return to his worksite.
There are a number of symptoms that can indicate the presence of fatigue, such as increased anxiety, decreased short-term memory, slowed reaction time, decreased work efficiency, reduced motivational drive, decreased vigilance, increased variability in work performance and increased errors of omission (Battelle Memorial Institute, 1998).
Work-related fatigue was assessed and was not considered to have contributed to the RSW’s actions. However, the RSW may have been experiencing some level of non-work related fatigue. The RSW commenced an overtime shift at 0500 on the day of the occurrence, this was the RSW’s third consecutive shift with the two previous shifts commencing at 0600. The RSW reported poor sleep the night prior due to a medical condition in the family. Additionally, he reported some broken sleep in the nights previous as well. This was not reported by the RSW when commencing work, as required by Sydney Trains’ fatigue management processes.
When considering the actions of the RSW, some actions resulted in increased risk but do not appear to be that of a person experiencing moderate fatigue, such as:
After entering the danger zone from Platform 10, they quickly moved to a safer area to walk along the cess within the rail corridor.
The distance walked by the RSW in attempting to return to their worksite does not indicate a reduction in motivational drive.
The RSW was quick to detect and respond to the approaching train. This does not indicate a reduction in reaction time or decreased vigilance.
Following the near miss, the RSW returned to Platform 10 via the same path taken.
On returning to the worksite, the RSW signed the project site induction indicating that the induction was received at 0900. The RSW was not present at this time and the time signed was incorrect. The RSW did not advise of the near miss incidents until after the completion of his rostered shift and after seeking medical attention for a cut hand.
Due to the late reporting, drug and alcohol testing could not be carried out. The RSW advised the ATSB they had not taken any medication or illicit substance prior to the incident. When the RSW reported the incident by telephone, he stated on a number of occasions he had been scared. It was not possible to determine why the RSW did not report the incident at the time, as required by Sydney Trains.
From the evidence available, the following findings are made with respect to the near misses involving the RSW and trains 89-K and 88-C that occurred near Redfern, New South Wales on 8 September 2018. These findings should not be read as apportioning blame or liability to any particular organisation or individual.
Safety issues, or system problems, are highlighted in bold to emphasise their importance. A safety issue is an event or condition that increases safety risk and (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.
Contributing factors
The RSW entered Redfern Station via the ESR without receiving or seeking a project site induction. The RSW then left the worksite to find a toilet.
The RSW attempted to return to the worksite after using the bathroom at Redfern, but did not seek assistance before entering the danger zone at Redfern Platform 10 or when sighting other workers on the access road.
The RSW entered the APL tunnel and not the ESR as intended and was nearly struck by trains 89-K and 88-C while in the APL tunnel.
The RSW was advised of the limit of the local area possession and knowingly entered the danger zone without any form of protection.
Sydney Trains' control of the access and egress to the project worksite did not ensure that all workers entering the worksite were identified and received an induction.[Safety issue]
Sydney Trains’ processes for cascading possession planning information did not ensure that all workers knew how to access the project worksite or the location of the site safety inductions.
Safety issues and actions
The safety issues identified during this investigation are listed in the Findings and Safety issues and actions sections of this report. The Australian Transport Safety Bureau (ATSB) expects that all safety issues identified by the investigation should be addressed by the relevant organisation(s). In addressing those issues, the ATSB prefers to encourage relevant organisation(s) to proactively initiate safety action, rather than to issue formal safety recommendations or safety advisory notices.
Depending on the level of risk of the safety issue, the extent of corrective action taken by the relevant organisation, or the desirability of directing a broad safety message to the [aviation, marine, rail - as applicable] industry, the ATSB may issue safety recommendations or safety advisory notices as part of the final report.
All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.
The initial public version of these safety issues and actions are provided separately on the ATSB website to facilitate monitoring by interested parties. Where relevant the safety issues and actions will be updated on the ATSB website as information comes to hand.
Sydney Trains’ Control of worksite access and egress
Safety issue description: Sydney Trains' control of the access and egress to the project worksite did not ensure that all workers entering the worksite were identified and received an induction.
Sydney Trains proactive safety action
Sydney Trains notified that the following proactive safety actions had been taken:
Signals support team were briefed on the incident and review of route knowledge for the City Underground and the ESR Line was included. Participants were also expressly advised that if they were unsure regarding working on these lines, they were to speak with their Work Group Leader or Team Leader to be familiar with the line prior to working on them.
Team briefings including lessons learned from the incident and reminders to Project Owners to conduct thorough risk assessments so that all site specific hazards are identified and adequate controls implemented. Project Owners were also reminded of the need to communicate changes affecting worksites in a timely manner and to all stakeholders.
Sources and submissions
Sources of information
The sources of information during the investigation included the:
Rail Safety Worker
Sydney Trains.
References
Battelle Memorial Institute (1998). An Overview of the scientific literature concerning fatigue, sleep, and the circadian cycle. Report prepared for the Office of the Chief Scientific and Technical Advisor for Human Factors, United States Federal Aviation Administration.
Dekker, S. (2011). Patient safety. New York: CR Press, pp.65-69.
Under Part 4, Division 2 (Investigation Reports), Section 26 of the Transport Safety Investigation Act 2003 (the Act), the Australian Transport Safety Bureau (ATSB) may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. Section 26 (1) (a) of the Act allows a person receiving a draft report to make submissions to the ATSB about the draft report.
A draft of this report was provided to Sydney Trains, the involved worker, Office of National Rail Safety Regulator (ONRSR) and Transport for NSW (TfNSW).
Any submissions from those parties will be reviewed and where considered appropriate, the text of the draft report will be amended accordingly.
Submissions were received from Sydney Trains and the Office of National Rail Safety Regulator. The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
A six-car Metro Trains Melbourne (MTM) passenger train travelling from Flinders Street Station to Lilydale Station uncoupled shortly after leaving Croydon Station. The train consisted of two 3-car units and uncoupled at the mid-coupling. The train was travelling at about 78 km/h when the 3-car units uncoupled. The trailing car unit came to a stop under automatic emergency braking. Once the train driver noticed that the trailing car unit had uncoupled, he applied emergency braking and brought the lead unit to a stop about 300 m from the trailing car unit. There were no injuries in the incident.
What the ATSB found
It was found that two technical faults had combined to cause the uncoupling. Inspection and testing conducted by MTM revealed that a wiring error was made during a modification to the train’s low-note whistle on the lead car of the train. Further, there was a deterioration of insulation resistance in the uncouple solenoid connector of the lead car of the trailing car unit. The low insulation resistance of the uncouple solenoid combined with the incorrect wiring of the low-note whistle resulted in an error circuit forming to energise and actuate the uncouple solenoid. This resulted in the unintended uncoupling.
What's been done as a result
The X’Trapolis fleet was inspected by MTM. Eleven of the motor cars were found to have the low-note whistles wired incorrectly and these were rectified. The low-note whistle wiring document has been updated to provide further clarity and to prevent future wiring errors with respect to this modification. In addition, maintainers involved in low-note whistle modification work have been briefed and re-trained on the modification method statements. Further, completed modification work will require self-checks and supervisor checks.
MTM advised that it has implemented a program of mandatory insulation resistance testing of the uncoupling solenoid connectors. The information from the testing will be provided to the original equipment manufacturer in order to assist with determining the root cause for the failure and take appropriate remedial action.
MTM further advised that it is reviewing the scope for improving negative to ground testing during depot based scheduled maintenance and exploring a design for a system for monitoring common negative to ground potential.
Safety message
This incident highlights the importance of ensuring that a verification program is incorporated into quality assurance systems to mitigate the risk of errors during installation or modification of electrical systems.
The occurrence
On the afternoon of 9 November 2018, two empty 3-car X’Trapolis units[1] were coupled at the Camberwell sidings in preparation for operating the 1607 Metro Trains Melbourne (MTM)[2] passenger service from Flinders Street Station to Lilydale Railway Station (Figure 1). Car 959M was designated the lead car of the train and operated as a non-revenue service from Camberwell sidings to Flinders Street Station.
Analysis of the VICERS[3] data logger indicated that at 1552, just prior to departure from Camberwell, the driver sounded the low-note whistle and there was a momentary loss of electrical signals between the leading and trailing 3-car units. At 15:52:15, the train departed from Camberwell sidings for Flinders Street Station.
On arrival at Flinders Street Station at about 1606, a driver exchange occurred. The new driver conducted the prescribed safety checks, entered the train describer number (TD 3817) and departed Flinders Street Station at about 1608. The train stopped at Southern Cross Station and Flagstaff Station before arriving at Melbourne Central Station at about 1615.
Figure 1: Melbourne train network showing train TD 3817 route on Lilydale Line
Source: Metro Trains Melbourne with annotations by Chief Investigator, Transport Safety (Vic)
The driver reported that at Melbourne Central Station, the message ‘Decoupling/Re-coupling’ appeared on the Driver’s Display Unit (DDU). He reported this to the Train Services Officer (TSO) at Metropolitan Train Control (METROL) who advised him that it could be the coupler pins not aligning in the centre coupler (coupler between the two 3-car sets). The driver reported that after a few seconds, a new message ‘PIDS[4] Inoperative’ appeared and then after a short period the normal screen was restored. He then re-entered the Train Describer number into the DDU.
Analysis of the data from the VICERS indicated an electrical disconnection lasting about 7 seconds at Melbourne Central Station. At about 1616, the electrical signal was re-connected, the emergency brake safety loop re-established and the train departed Melbourne Central Station.
The train stopped at several stations before arriving at Croydon Station at about 1656. Although the DDU did not indicate any further irregularities, the VICERS data download traces between Melbourne Central Station and Croydon Station indicated three further electrical disconnections. These electrical disconnections did not disrupt the operation of the train.
The train departed Croydon Station at about 1657, with about 150 passengers on board. After leaving Croydon Station and on the approach to Coolstore Road level crossing (about 200 m from the station) the driver sounded the low-note whistle. The train then entered a left-hand curve, about 800 m from Croydon Station. About half way through the curve, the driver looked back through the rear-view mirror and noticed that the train consist had only three cars and the trailing unit had uncoupled (Figure 2). He immediately applied emergency braking and brought the three-car set to a stop.
Figure 2: Coupled cars 960M and 882M of X’Trapolis train that uncoupled
Source: Metro Trains Melbourne annotated by Chief Investigator, Transport Safety
The VICERS data download indicated that about 30 seconds after departing Croydon Station, an emergency brake application was made on cars 882M and 881M, and the trailing 3-car set came to a stop at about 1658. The download further indicated a momentary brake application on cars 959M and 960M (the motor cars of the leading 3-car set), but this brake application was not sustained and the car set continued moving. The VICERS download indicated an emergency brake application at about 1658 and shortly after, the leading car set 959M came to a stop about 371 m from the lead car 882M of the trailing car set. The train was travelling at about 78 km/h when the 3-car sets uncoupled. The trailing car set travelled under emergency braking for about 196 m before coming to a stop.
Events after uncoupling of 3-car units
After the leading car set came to a stop, the train driver made a priority radio call to advise METROL of the uncoupling of the trailing car set. METROL despatched an empty train (under extreme caution[5]) from Lilydale Station towards Croydon Station to locate the 3-car set that had detached from the lead car set. Victoria Police, the Melbourne Fire Brigade (MFB) and the Ambulance Services arrived soon after and detrained about 150 passengers from the two detached 3-car sets.
The parting of the two 3-car units of train TD 3817 occurred about 500 m from Croydon Station. Croydon Station is on the Lilydale rail line and is located about 32 rail km from Flinders Street Station, Melbourne (Figures 1 and 3).
Figure 3: Location of Croydon on Lilydale train line
Source: Copyright Melway Publishing 2018, with annotations by the Chief Investigator, Transport Safety (Vic)
The environment and track
At the time of the incident, the weather was fine with light winds and the temperature was about 190 C.
The rail track from Croydon Station to the location of the train parting was concrete sleepered. From Croydon Station there were a number of left and right hand curves between 31.2 km and 32 km marks. The track slopes down at a slight gradient from 31.2 km to about 31.5 km and then is flat until about the 32 km mark.
Train driver
The train driver was employed by MTM for over 9 years and was qualified to drive Electrical Multiple Units (EMU). His driving performance was audited regularly by MTM driver supervisors and was last audited in July 2018. No non-conformances were recorded during these audits. Medical certification for the driver was valid and current at the time of the incident. No alcohol or drugs were detected during post-incident tests conducted on the driver.
The train
The X’Trapolis EMU is operated as a 3-car or 6-car set. The car body modules were designed and constructed in France and assembled at Alstom Australia, in Ballarat, Victoria. Each 3-car set is coupled in motor-trailer-motor configuration. They are coupled by auto-couplers between motor cars and semi-permanent couplers between the motor and the trailer cars, with rubber bellows inter-car canopy walkways.
TD 3817 was a 6-car set and consisted of 959M-1680T-960M-882M-1641T-881M passenger cars.[6] A Scharfenberg automatic coupler connected the two 3-car units of TD 3817 between 960M and 882M. A 6-car train set has the total effective capacity for 752 passengers and this train was carrying 150 passengers at the time of the uncoupling.
The motor units are 24.46 m in length and the trailer units are 22.76 m in length. Each 3-car set is 71.68 m in length and a 6-car set is 143.36 m in length. The average mass of a 6-car set is about 244 t. Each car has six doorways, located at the front, middle and rear of the cars and the nominal floor height of the car is 1185 mm above top of the rail. The EMU is capable of a maximum speed of 130 km/h.
Master Controller and Operations
In order to prepare the train for operation, the driving cab must be made electrically operational by inserting and switching on the Master Key making it the cab-in-service. The driver then enters the Train Describer number into the radio control unit. The Master controller handle is moved back and forth between four positions—motor, coast, brake and emergency brake. The Reverser is a three-positon switch that can be moved to forward, neutral and reverse positions. In order to get forward traction, the Reverser must be moved to the forward position and the master controller moved from the brake to coast to the motor position. The position of the Master controller between the coast and motor positions dictates the tractive effort. The console and controls are accessible from the driving position (Figure 4).
Figure 4: Master Controller Panel
Source: Chief Investigator, Transport Safety (Vic)
Vigilance, Control and Event Recording System (VICERS)
The X'Trapolis fleet is fitted with a Vigilance Control Event Recorder System (VICERS). The vigilance component monitors task-linked activities and in the absence of any such activities provides intervention by applying the train’s brakes. It also incorporates an event recorder system, which monitors a number of on board systems including the operation of a number of driver control functions. The system records a comprehensive range of operational parameters including control equipment status, train speed, traction and braking and location.
Coupling and Uncoupling of Scharfenberg Coupler
Each end of the 3-car units were equipped with an automatic Scharfenberg coupler that enabled mechanical, pneumatic and electrical coupling of two 3-car units (Figures 5 and 6).
Figure 5: Mechanical, pneumatic and electrical components of Scharfenberg coupler
Source: Chief Investigator, Transport Safety (Vic)
Figure 6: Scharfenberg coupler between cars shown uncoupling
Source: Chief Investigator, Transport Safety (Vic)
The Scharfenberg coupler has a coupler lock that consists of a rotating hooked plate, a coupling link and tension springs (Figure 7). During coupling, the coupling link of one coupler engages the hooked plate of the mating coupling and locks in the indentations on the hooked plates. The cone and funnel shape design of the coupler front plates establishes a rigid and slack-free connection. During the coupling, the electric heads and the air pipe connections engage to ensure connectivity between the two car units. The coupler has a flexible buffer assembly that allows movement of the cars when negotiating vertical and horizontal curves.
Figure 7: Illustration of uncoupled Scharfenberg coupler
Source: Voith Group, annotated by Chief Investigator, Transport Safety (Vic)
The units can be coupled or uncoupled remotely from the driver’s cab or manually from trackside. The automatic coupling process involves moving one 3-car unit against a stationary 3-car unit.
X’Trapolis electrical and pneumatic systems for uncoupling
The Melbourne metropolitan electrified rail network operates on 1,500 V direct current (DC), supplied to the train via a roof-mounted pantograph that is in contact with the overhead wire. Two power inverters (static) convert the 1500 V DC to 400 V, 3-phase or 230 V single-phase alternating current (AC) to supply the various loads on the train. The 400 V AC supply is rectified and transformed to provide 120 V DC through a battery charger system.
The 120 V DC power supply is distributed to three lines; the permanent LV line, prepared LV Line and the prepared and cab operative line, each corresponding to an operational state of the train.
The prepared LV line supplies the uncoupling system (Figure 8). An intended uncoupling operation requires the train to be stationary and local cab active. The uncouple pushbutton (Figure 4 and 8) is pressed to energise and actuate the uncouple solenoid valve (VS). The energised uncouple solenoid valve directs supply air to the uncouple cylinders (Z3) on both couplers
Figure 8: Wiring diagram for uncoupling system
Source: Alstom Drawing AX00000263675, simplified and annotated by Chief Investigator, Transport Safety (Vic)
The uncouple cylinder (Z3) actuating piston rotates the hooked plates of the coupler against the force of the tension springs (Figure 7 and 9). The coupling links slip out of the indentations on the hooked plates and the couplers can then be separated. The uncouple cylinder actuating piston also rotates the cam (C1) that actuates the 5/2 way pneumatic valve (V4). Actuation of the 5/2 way pneumatic valve allows air to be directed to the pneumatic cylinders (Z4) that retract the electrical heads (E).
For the uncouple pushbutton to activate the uncouple solenoid valve, the cab has to be in-service (cab active) and the train speed has to be below 3 km/h (Figure 8). Pressing the uncouple pushbutton from the active cab energises the uncouple command relay which deactivates the coupling detection contactor relays. These in turn disconnect the traction and braking signals across the coupler electrical heads prior to uncoupling.
Figure 9: Pneumatic schematic for uncoupling of Scharfenberg coupler
Source: Scharfenbergkupplung GmbH & Co. (annotated by Chief Investigator, Transport Safety (Vic)
Metro Trains Melbourne (MTM) conducted multiple dynamic tests of the train in an attempt to replicate the fault, without success. The Scharfenberg coupler and associated mechanical and pneumatic components were inspected and no fault was identified. The Scharfenberg coupler can be manually uncoupled by actuating the automatic coupler manual release handle. Examination of the manual release mechanism did not indicate that it had been used. It was concluded that the most likely initiator for an uncoupling event was via the activation of the uncouple solenoid (22Y3).
Insulation resistance testing of uncouple solenoid
Uncouple solenoid connectors of the coupled motor cars (959M and 882M) were tested for insulation resistance (IR)[7] to ground (vehicle body/frame). The IR value for the uncoupling solenoid connector of car 882M was found to be 20 kΩ, when the typical IR is required to be over 100 MΩ. However, the low IR would not result in unintentional energising of the uncoupling solenoid valve without a power source to the solenoid.
Source: Chief Investigator, Transport Safety (Vic)
Further, testing indicated that both the uncoupling solenoid and the uncouple relay picked-up and actuated at a minimum voltage of about 50 V, which was not unusual.
Insulation resistance testing carried out on all the uncoupling solenoid connectors of the X’Trapolis fleet revealed that 15 uncoupling solenoid connectors measured low insulation resistance. In addition, 19 trip re-set connectors[8] and 15 trip raise connectors[9] also indicated low insulation resistance.
Low-note whistle wiring error
MTM conducted test runs of the 3-car set with car 959M as the lead cab-active car. Normally, zero voltage should exist between a power supply common negative and ground (earth/vehicle body). However, during the tests on car 959M, an oscilloscope measured an intermittent 120 V between the common 120 V negative and ground. Examination of the VICERS download trace indicated that the voltage spike coincided with the use of the low-note whistle (Figure 4). Further, static testing confirmed that activation of the low-note whistle resulted in the application of 120 V supply between common negative to ground.
Examination of the low-note whistle wiring indicated that a relay on car 959M was installed incorrectly and a cable was connected to a wrong terminal. This error resulted in the low-note whistle’s 120 V supply being connected to the negative (24 V) of the Speed Interface Unit (SIU), which is an interface between the car’s speed detector and the VICERS unit (Figure 11).
MTM’s investigation of the wiring error revealed that although the electrical circuit diagrams were accurate, the technician carrying out the modification had made a cable connection error. Inspection of the X’Trapolis fleet indicated that 11 low-note whistles were wired incorrectly by the same technician. A function test would not reveal the wiring error, as pushing the low-note whistle button would complete the circuit to actuate the low-note whistle. In this instance, an assurance program involving a secondary and independent check of the wiring modification may have been the only means of detecting this error.
Figure 11: Electrical schematic showing incorrect wiring of low-note whistle
Source: Chief Investigator, Transport Safety (Vic)
Speed Interface Unit (SIU) design
The SIU converts the speed signal (0-20 mA) to a pulse train output to the VICERS unit. It provides isolation between the train circuits and VICERS unit. The negative of the SIU was connected to ground providing an electrical ground for the 24V system and the circuit path to actuate the uncoupling solenoid valve.
The safety loop circuit and uncoupling
The safety loop is a control circuit passing through each carriage within the trainset and is closed during operation under normal conditions. When the safety loop is opened due to an abnormal operating condition, the emergency brake solenoid valve is de-energised causing a maximum brake application.
In this instance, the uncoupling function was activated through an electrical circuit that was created by a wiring error in car 959M and the low IR of the uncouple solenoid of car 882M. This error resulted in the cab active relay, push button and the 3 km/h relay interlock devices being bypassed and allowing the unintended activation of the uncoupling circuit. The safety loop circuit on the lead car unit maintained the closed safety loop, hence emergency braking was not applied and it continued its journey until the driver realised that the trailing car unit had uncoupled, and brought the lead 3-car unit to a stop. As there were no active cabs in the trailing 3-car unit, the safety loop was opened, de-energising the emergency brake solenoid valves, resulting in the application of the emergency brakes on the trailing car unit bringing it to a stop.
The following findings are made with respect to the parting of a passenger train TD 3817 at Croydon, Victoria on 9 November 2018. These findings should not be read as apportioning blame or liability to any particular organisation or individual.
Safety issues, or system problems, are highlighted in bold to emphasise their importance. A safety issue is an event or condition that increases safety risk and (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.
Contributing factors
A wiring error was made during a modification to the trains low-note whistle on car 959M.
The wiring error was not detected by Metro Train Melbourne’s verification program. [Safety Issue]
There was a deterioration of insulation resistance in the uncouple solenoid connector of car 882M.
The connection between the negative and ground via the Speed Interface Unit provided a path that created the circuit to actuate the uncoupling solenoid valve.
Safety issues and actions
The safety issues identified during this investigation are listed in the Findings and Safety issues and actions sections of this report. The Australian Transport Safety Bureau (ATSB) expects that all safety issues identified by the investigation should be addressed by the relevant organisation(s). In addressing those issues, the ATSB prefers to encourage relevant organisation(s) to proactively initiate safety action, rather than to issue formal safety recommendations or safety advisory notices.
Depending on the level of risk of the safety issue, the extent of corrective action taken by the relevant organisation, or the desirability of directing a broad safety message to the rail industry, the ATSB may issue safety recommendations or safety advisory notices as part of the final report.
All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.
Safety issue description: The wiring error was not detected by Metro Train Melbourne’s verification program.
Additional safety actions
Whether or not the ATSB identifies safety issues in the course of an investigation, relevant organisations may proactively initiate safety action in order to reduce their safety risk. The ATSB has been advised of the following proactive safety action in response to this occurrence.
MTM advised that it has implemented a program of mandatory insulation resistance testing of the uncoupling solenoid connectors. The information from the testing will be provided to the original equipment manufacturer in order to assist with determining the root cause for the failure and take appropriate remedial action.
MTM further advised that it is reviewing the scope for improving negative to ground testing during depot based scheduled maintenance and exploring a design for a system for monitoring common negative to ground potential.
Sources and submissions
Sources of information
The sources of information during the investigation included:
Metro Trains Melbourne Pty Ltd.
SNC-Lavallin Rail & Transit Pty Ltd.
Submissions
Under Part 4, Division 2 (Investigation Reports), Section 26 of the Transport Safety Investigation Act 2003, the ATSB may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. Section 26 (1) (a) of the Act allows a person receiving a draft report to make submissions to the ATSB about the draft report.
Submissions were received from Metro Trains Melbourne Pty Ltd and the Office of the National Rail Safety Regulator. The submissions were reviewed and where considered appropriate, the text of the report was amended accordingly.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
Occurrence summary
Investigation number
RO-2018-019
Occurrence date
09/11/2018
Location
Croydon, 32 km from Flinders Street Station, Melbourne
On the morning of 21 September 2018, a train driver was using remote control equipment (RCE) to control and position TasRail train no. 604 so that it could be loaded with cement powder at a siding in Railton, Tasmania. The driver was outside the train and no-one else was on board.
Having overshot the intended location to load the last 2 wagons, the driver attempted to use the RCE transmitter to reverse the train. However, the train did not respond to their commands. The driver attempted procedures to recover the RCE from known failure modes in order to restore control, however these were unsuccessful.
As the driver began to walk towards the locomotive to attempt to reset equipment, they noticed the train start to move. They attempted several methods to command the train to stop by using the RCE transmitter, but these were unsuccessful. Instead, the train, which was on a downhill grade, continued to roll away, leaving the siding and entering the main line toward Devonport.
The driver notified TasRail network control of the runaway, and they in turn notified emergency services. Network control also organised for the points at the entrance to Devonport Yard to be set to divert the train from the main line, which continued past this location, into a dead-end siding within the yard.
The train travelled through 10 active and 3 passive public level crossings, beneath a highway overpass, and through 5 sets of points, mostly at speeds greater than the maximum track speed. It reached a maximum recorded speed of 87.5 km/h.
About 23 minutes after the train rolled away from Railton, it collided with a concrete footing and surrounding fences at the end of the silo siding in Devonport Yard. The derailment caused significant damage to the train’s locomotive and 7 wagons, as well as damage to the end of the siding and fences. Fence debris struck 2 pedestrians who had been walking in the area, resulting in minor injuries to both.
What the ATSB found
Key parameters related to the remote control equipment (RCE) were not recorded. This hindered internal and external safety investigations of events involving use of the RCE, both for this accident and previous events. However, the ATSB was able to determine that the RCE entered a spurious fault condition (involving the application of emergency braking) that was almost certainly associated with rapid movement of the RCE transmitter’s direction controller by the driver. This likely initiated an unintended interaction between a safety feature (the dual direction fault interlock) and braking systems on the locomotive, which resulted in an emergency-level brake application.
At about this time, the RCE ceased to respond to driver commands. Instead, the initial brake application was released, then 2 further emergency brake applications and releases occurred. These did not replicate any documented fault condition behaviour for the RCE. A short time later, while still unresponsive to driver commands and with the train’s brakes still released, the train started to roll away.
When a driver was operating the train from outside the train (such as during loading and unloading), there was no way for them to stop the train if the RCE ceased to respond. Also, the TasRail cement loading facility at Railton had a downhill grade to the main line, and no devices to protect against runaway.
The RCE receiver was designed to apply the train’s brakes when it was outside of radio communication range with the transmitter, but this did not occur when the train rolled away from Railton. Instead, the train proceeded to roll away with the brakes remaining released until it derailed in Devonport Yard.
When network control was notified of the runaway, TasRail staff and emergency services conducted a prompt and effective response. This included routing the train away from the main line into the yard and protecting some level crossings; actions that minimised the risk exposure of the public. However, the successful response on this occasion was not assured, as TasRail’s processes for responding to emergency situations fundamentally relied on the experience and knowledge of NCOs and did not include the provision of procedures, tools and checklists detailed enough to support the effective management of specific types of incidents that require a time-critical response.
The ATSB examined a range of topics associated with the development of the RCE, and its use more broadly. At the time of the runaway, TasRail were using the third generation of RCE developed and manufactured by Air Digital Engineering (ADE) to remotely operate the cement train. The generation 3 RCE had been commissioned for use in February 2018 and incorporated several changes from the previous generation used by TasRail. However, the generation 3 RCE had several safety-related design and integration problems, including the potential for unintentional activation-and-release of emergency braking on the locomotive and opposition to externally-initiated emergency brake applications. These issues existed at the time of the accident but had not been identified during the testing and commissioning processes.
The RCE was used as the primary driver control interface to control the train, both when loading/unloading with the driver outside and during transit when the driver was either in the locomotive or the driver’s van. As such, the function that the RCE performed was safety critical. However, although ADE had safety as a design objective and safety elements were included in the RCE, systems safety assurance activities appropriate to its application were not conducted.
TasRail had operated the cement train service successfully using previous generations of the RCE for about 19 years without any accidents attributable to the RCE, which likely contributed to a high level of confidence in the RCE and overall operation. Nevertheless, the development and integration of new equipment is not solely the responsibility of the equipment manufacturer; the organisation using the equipment also has a duty to ensure it is safe. In this case, TasRail commissioned the manufacture of the new equipment without fully engaging with the development process or understanding the extent of design changes since the previous generation of RCE, identifying and imposing safety requirements, or verifying that the overall system met a specified level of safety.
The ATSB also found that there were problems associated with TasRail’s management of change. Most activities that were required by TasRail’s procedures for a significant change were not performed during the project to develop and implement the generation 3 RCE. Furthermore, the change management process, while detailed, had limited capability to assure that activities, approvals, and documentation that were identified as being required were completed. The process also had limited capability to reliably determine whether the change being considered had the potential to impact safety or identify what safety assurance activities were needed to manage the change.
In addition, there were broader safety issues identified relating to TasRail’s remotely-controlled train operations, separate to the implementation or use of the generation 3 RCE. TasRail had not identified or fully assessed the safety implications of remotely-controlled train operations or those of its specific implementation. TasRail also did not have a reliable process to systematically identify, track and analyse reported faults on its remotely-controlled train or to identify their potential safety implications.
The ATSB also identified that, at the time, there was limited practical guidance specifically for the Australian rail industry for the application of system safety assurance processes to the development of complex and safety-critical rail systems. Although standards, legislative requirements, and guidance for rail safety were recognised and applied in Australia, they were of limited value in the development and operation of complex systems.
There was also no explicit regulatory requirement for developers of rail equipment to demonstrate an objective evaluation of design safety or apply system safety principles during development. If a rolling stock operator or equipment developer intended to apply system safety principles, there were numerous challenges in attaining justified confidence in its success, which were difficult even for experts to overcome. Overall, the standards and guidance for system safety available at the time of the runaway were too abstract, complex, costly and/or impractical for widespread recognition and acceptance by the Australian rail industry.
Separately, there is a requirement for rolling stock operators to notify ONRSR when they intend to make certain changes. A change to a safety critical component, such as the RCE, should be considered the type of change which should be notifiable to ONRSR, so that regulatory oversight may occur. However, TasRail did not submit a notification of change for the generation 3 RCE and this was likely because an accepted view had formed within TasRail that it was ‘like-for-like’ replacement and therefore minimal change was occurring.
Guidance material prepared by ONRSR relating to the regulatory requirements for notifying change included limited detail about the extent or type of changes that necessitated a notification. One category that was identified as requiring notification was ‘a safety critical element of rolling stock’. However, the guidance did not provide an interpretation of ‘safety critical’ or the applicability of the requirement to equipment that may not be inherently part of rolling stock (such as RCE). In this case, a notification of change had not been provided to ONRSR, but the reasons for this were not found to be related to the interpretation of the regulation.
What has been done as a result
Following the accident, TasRail immediately suspended remotely-controlled train operations. In November 2020, TasRail completed installation of a catch point at the western end of the Railton siding to prevent runaway rail vehicles from entering the main line. TasRail also:
extensively revised its processes and guidance on change management, project management, engineering design
enhanced processes for fault tracking and analysis
implemented checklists to help ensure consistent and sound decisions during time-critical emergency responses.
ADE advised that it would re-evaluate the generation 3 remote control equipment under system safety design principles if it were to be used for future operations.
ONRSR advised that it planned to provide additional guidance about interpreting the conditions that describe when a notification of change to ONRSR would be required.
Other safety actions have also been taken that help address safety issues identified by the ATSB (although were not initiated as a direct result of this accident):
In November 2018, the Rail Industry Safety and Standards Board (RISSB) finalised Australian Standard (AS) 7472 (Railway operations - management of change), to assist rail transport operators in fulfilling change management responsibilities. (This standard was in development prior to the accident.)
ONRSR published a safety message in March 2019 titled Importance of a System Engineering Approach and 2 related fact sheets in 2020 that emphasised the need for a systems engineering approach in rail projects and drew attention to the relevant standards. The safety message stated that rail transport operators ‘should demonstrate that they use appropriate systems engineering processes and safety assurance processes… in their design and procurement approach.’
In 2020, RISSB issued standard AS 7473 (Complex system integration in railways) to support a systems engineering approach in Australian rail projects.
In 2021, RISSB issued standard AS 7474 (Rail industry – System safety) to ‘provide a clear standard for management of System Safety that addresses Australian legislative requirements and is readily scalable for the scope of rail projects undertaken within Australia.’
In addition to providing an overview of the relevant methods, AS 7473 and AS 7474 referred to other more detailed systems and safety engineering standards to enhance their visibility and application across the Australian rail industry.
Safety message
Rolling stock operators and equipment developers both have duties to ensure the design and modification of rolling stock and other equipment is safe for its intended use. This accident reinforces the need for such organisations to identify what systems are performing safety-critical functions and manage the risks of systems failure by ensuring the system has appropriate levels of reliability and/or by using secondary safety components that can override a failed system. This is especially true of complex systems such as those that implement software, due to the potential for these systems to fail in ways that are hard to predict.
Systems for the management of change need to reliably assure the determination of whether a change had the potential to impact safety and identify the need for relevant safety assurance activities. When a new system is implemented, or changes occur to a system, effective safety management needs to be a joint effort between the operator and the developer in order to effectively identify safety hazards and ensure that associated risks are managed. Operator involvement at this stage is important to fully account for the safety implications of the broader system.
Overall, it is important to ensure that systems have a compelling argument for their safety. Developing such an argument is not a simple task but there are numerous standards and guidelines available for a range of applications, including (more recently) specifically for the Australian rail industry. The ATSB strongly advises rolling stock operators and developers of rail systems to apply system safety methodologies so that they can have a high level of confidence in the overall safety of the system. The ATSB additionally encourages the continuing development of guidelines and standards to enhance the accessibility and practical application of system safety principles for all rail operations.
In addition, this investigation has also highlighted that rolling stock operators should ensure that event recorders are recording sufficient parameters for optimal outcomes of safety investigations, including those conducted by the operator.
The occurrence
Overview
TasRail operated train services between Railton and Devonport, Tasmania, for the purpose of transporting cement powder. The trains were operated by a single driver using remote control equipment (RCE).[1] The trains consisted of a TR class locomotive, cement wagons and a driver’s van.
When a train was being loaded with cement at the Railton loading facility, the driver was outside the train and no-one else was on board. The driver operated both the RCE and the loading equipment to transfer the cement powder from silos via chutes into the train’s wagons.
At 0846[2] on 21 September 2018, train no. 604 was being loaded. While the train driver was attempting to manoeuvre the last 2 wagons into position for loading, the train did not respond to remote control commands from the driver. The train (which was on a downhill grade) then rolled away from the facility without any train crew on board and in an uncontrollable state. The train driver informed the network control officer (NCO) about the runaway train, and then arrangements were made for it to be rerouted away from sensitive infrastructure.
The train travelled for about 20 km before reaching a dead-end siding in Devonport at 0909. The train collided with the end of the dead-end siding, and travelled in a derailed state for about 60 m into a public area.
Operations before the occurrence event
At 0130 on 21 September 2018, the driver signed on at Devonport Yard to work the service between Devonport and the Railton cement loading facility, taking over from another driver who had worked the preceding shift. The drivers performed a handover at the cement silos at Devonport where train no. 512[3] was being unloaded, with the previous driver advising that there were no faults or issues with the train or the RCE.
At 0220, the driver departed with the empty train from Devonport for Railton, where the 16 wagons were to be loaded. The train returned to Devonport to unload, before returning to Railton at 0740 to load again. During these cycles between Devonport and Railton, the RCE and train continued to perform as expected, with no faults or issues identified.
On arrival at Railton, the driver alighted from the train in preparation for checking the wagon doors prior to loading the train (now train no. 604). The Railton cement siding was located on the southern side of the main line between Devonport and Railton Yard. The loading facility (consisting of 2 cement powder silos) was situated on the siding (Figure 1). From the loading facility, the track had a downhill grade of 1 in 139[4] westward towards Devonport. Access to the main line was through a set of trailable electric points.
Figure 1: Railton cement siding and loading facility
Source: TasRail, modified by the ATSB
Loading train no. 604
At about 0746, after checking the bottom discharge doors on the wagons as the train passed through the loading facility towards Railton Yard, the driver aligned wagons 1 and 2 under the silos in preparation for loading. As the loading process was designed to be a single person operation, the driver then entered the loading facility and proceeded to the upper level where the loading control panel was positioned (see Train loading process).
The driver removed the key from the RCE transmitter and placed it into the loading control panel to allow operation of the panel’s loading controls.[5] The driver then commenced loading of the first 2 wagons with cement powder.
Once the 2 wagons were full, the driver removed the transmitter key from the loading control panel and placed it back in the RCE transmitter, allowing driver commands to be sent to the RCE receiver on the locomotive. The driver then moved the train forward towards Devonport using the transmitter, aligning the next pair of wagons for loading. This process continued for the first 14 wagons (that is, the first 7 placements).
At about 0841, the driver removed the transmitter key from the loading control panel and placed it into the RCE transmitter to facilitate the next train movement forward to load the final 2 wagons. Using the transmitter, the driver released the automatic (train) brake, placed the direction controller in forward, released the independent (locomotive) brake, and applied light traction power. As the train came to a stop, the driver noticed that they had misjudged the wagon alignment, overshooting the stopping mark[6] by about 20 cm.
Remote control equipment failure
The driver recalled that, after the train came to a stop, they requested the direction to change (via the RCE transmitter) in preparation to reverse the train. The data recorded on the locomotive showed that the direction went immediately from forward to reverse without registering in neutral.
The driver informed the ATSB that, after releasing the train’s brakes, they applied traction power to facilitate the rearward realignment of the wagons. After about 1 minute, allowing for the train’s brakes to settle and traction power to take effect, the driver noticed that the train was still not moving. The driver stated that the RCE transmitter display screen was showing the text ‘Connecting 9863 v1.4’, instead of the usual system status information.
On observing the displayed message, the driver believed that the RCE had a communications failure,[7] so they configured the RCE transmitter controls for a reset (direction in neutral, throttle idle, independent brake fully applied, and automatic brake set) and pressed the train set button to reset the fault. The displayed message did not clear as it normally would.
Given the lack of response, the driver then believed that the RCE was experiencing a control fault.[8] They reported following the procedure to reset the control fault mode, which was to:
retain the transmitter configuration as described above
turn the transmitter’s key switch off, wait about 10 seconds
turn the transmitter key switch on again and press the train set button to return to ‘run’ mode.
The displayed message immediately reappeared on the transmitter display screen, so the driver repeated the process. Immediately after the second attempt at a control fault mode reset, the same message reappeared, so the driver determined a ‘cold start’ reset would be required to clear the fault. This required the driver to proceed to the rear of the locomotive (14 wagons away) to disconnect and reconnect the multiple unit cable[9] between the RCE receiver and the locomotive.
The driver informed the ATSB that the transmitter display screen continued to display the message ‘Connecting 9863 v1.4’ throughout the initial occurrence sequence. This indicated that the driver’s RCE transmitter did not have a communication connection to the RCE receiver on the locomotive.
The runaway
At 0846:42, just as the driver had started to walk along the loading platform (while carrying the RCE transmitter) to perform an RCE reset at the locomotive, they noticed the train start to move forward. As the independent (locomotive) brake was still applied on the RCE transmitter, the driver thought that the train was momentarily ‘bunching’ coupler slack[10] on the downhill grade against the stationary locomotive.
The driver quickly realised that the train was moving past the expected extent of a bunch movement and they applied a full-service automatic (train) brake application on the RCE transmitter. When that did not work, they turned off the transmitter key switch to cause a communication failed mode automatic brake application. When the train still did not respond, the driver tilted the RCE transmitter[11] and then removed the battery as further attempts to command a brake application, which were also ineffective.[12] The driver also stated that they thought they had tried to use the ‘emergency stop’ switch on the RCE transmitter but did not recall when in the sequence this occurred.
The driver observed the driver’s van at the rear of the train exit the Railton loading facility in the direction of Devonport. Believing that the RCE’s vigilance function[13] would automatically apply the brakes, the driver crossed the loading platform bridge and walked down the stairs to observe the train’s stopping position. The train continued to roll out of the cement siding and onto the main line out of the driver’s view. With no further method to stop the train available, the driver immediately rang the network control centre to report the runaway train.
Emergency response
At 0848:05, the network control officer (NCO) answered the driver’s phone call and was advised that there was a runaway train. The NCO immediately checked the section of track from Railton to Devonport was clear and discussed with the driver where the train was likely to stop. It was determined that this could be prior to Devonport at Horsehead Creek, due to a rising grade and 40 km/h curves. In a scenario where the train continued to roll past Horsehead Creek, background conversations of other TasRail staff in attendance determined that the best course of action was to set the points to divert the runaway train into Devonport Yard, away from the main line and several level crossings. There it would derail at a dead-end siding, away from the township.
To facilitate the switch in points, the general manager freight services (who had been alerted to the runaway and was able to assist) contacted the shunter based at Devonport at 0852 and instructed them to set the points (no. 85 and no. 88) for no. 2 road Devonport (silo siding) (Figure 2). When the shunter advised this had been done, the general manager instructed the shunter to warn local businesses near the yard to evacuate.
Figure 2: Devonport Yard showing the route taken by the train to the dead-end siding
Source: TasRail, modified by the ATSB.
At 0853:52, 1 minute after the phone call with the driver concluded, the NCO contacted emergency services. The NCO advised Tasmania Police of the runaway event and the location of major level crossings between Spreyton and Devonport that would require police protection, with derailment now deemed likely to occur at the end of Devonport Yard.
The TasRail rolling stock asset manager had been alerted to the runaway. They logged into a computer in a room near to but separate from the network control centre to view the train’s GPS speed and location information to allow monitoring of the train in real time. They relayed this information verbally to the network access manager, who was present in the network control centre at the time the runaway train was reported by the driver.
At 0858:41, the network access manager took over from the NCO communicating with the police to streamline the relay of messages. Over the next 11 minutes, the network access manager liaised with emergency services to deal with the emergency, including:
providing regular GPS train speed and location updates
advising likely derailment points between Latrobe and the entry to Devonport Yard
discussing the possibility of a derailment at no. 88 points near the Bass Highway causing a bridge pylon strike
requesting fire services and ambulances to attend Devonport Yard
advising the final stop location after the train had derailed.
The train travelled through 10 active and 3 passive public level crossings, beneath a highway overpass, and through 5 sets of points,[14] mostly at speeds greater than the maximum track speed at each location. It reached a maximum recorded speed of 87.5 km/h. A summary of the train’s location and speeds during key events of the runaway sequence are summarised in Track features.
At about 0900, Tasmania Police despatched 20 police units to respond. The police response focussed on protecting the public in front of the train by alerting pedestrians and vehicles to the danger through activation of lights and sirens on all attending police vehicles and the closure of level crossings. However, not all road-rail interface points could be protected in time, including the Bass Highway overpass.
The derailment
At 0909:29, the cement train collided with a concrete footing and fences at the end of the silo siding in Devonport Yard. The train derailed, and then came to a stop adjacent to the Mersey River Torquay Ferry pontoon in Pioneer Mariner’s Park.
Shortly before the derailment, 2 pedestrians who were walking in the area heard the police sirens and looked up and saw the oncoming train. Although they quickly moved away, they were both struck by fence debris, resulting in minor injuries.
In addition to the damage to the end of the silo siding, fences and park surrounds, 7 cement wagons and the locomotive were significantly damaged, with wagon wreckage blocking the main line between Devonport and Burnie (Figure 3).
Figure 3: Derailment site in Devonport, Tasmania
Image shows the location of the injured pedestrians at the time of the derailment, as well as the cement train consist configuration. Source: ATSB
Following the accident, TasRail suspended RCE operations. The cement train service recommenced 2 days after the accident with a locomotive at both ends and with a 2-person crew configuration. The main line between Devonport and Burnie reopened to trains at about 2300 on 24 September 2018.
Additional occurrence sequence information
The ATSB’s analysis of information relating to the accident sequence (including during loading, the runaway, and the derailment) is presented in Appendix A. The analysis drew on information from multiple sources, including data recorded on the locomotive, the driver’s recollection, and both data and observation of system behaviour obtained during subsequent testing.
The TR class locomotive and the driver’s van [15] were fitted with data recording devices (see Event recorders for more information). The RCE had no data recording capability, which limited the investigation’s ability to identify or understand some of the key events.
Based on the available evidence, including the data recordings, the ATSB analysis identified that the following occurred during the accident sequence:
The RCE became unresponsive to driver commands at the point of, or soon after, a fast direction change from forward to reverse.
The RCE initiated a momentary emergency vent of the brake pipe, likely due to a momentary ‘dual direction fault’ being detected. The emergency vent resulted in the locomotive’s VX vent valves and emergency magnet valve (EMV) activating to exhaust the brake pipe further, which the RCE tried to oppose after the fault state cleared by supplying pressurised air from main reservoir no. 1 to the brake pipe.
The RCE triggered 2 further emergency brake applications and releases.
The RCE ultimately entered an unsafe state with the brakes on the train and locomotive fully released, allowing the train to roll away on the downhill grade towards Devonport.
The RCE did not command a ‘communication failed mode’ or apply the train’s automatic brake, either when the transmitter and receiver were unlinked, or once the receiver was out of radio range of the transmitter.
The emergency brake applications and releases, and subsequent state with brakes released without a radio link, were not consistent with any documented fault condition behaviour.
The RCE remained in an unsafe state, which caused the brakes on the train and locomotive to be maintained in a fully-released condition, until the train’s brake pipe was ruptured in the derailment.
After comprehensive testing and analysis, the ATSB also determined that:
The locomotive, including its control and safety systems, operated as designed.
The locomotive electronic airbrake system operated as designed.
There was no evidence of malicious interference to the RCE or train systems.
Given this pattern of evidence, the ATSB concluded that the RCE entered a spurious fault condition (which applied the emergency brake) when the driver rapidly commanded a direction change, and then entered a persistent unresponsive state in which the train’s brakes were not applied.
This unresponsive state was not reproduced during post-accident testing activities and the specific mechanism involved was not identified. Nevertheless, the fact that the unresponsive state occurred on this occasion indicated that a latent failure mode existed within the RCE system that could not be readily diagnosed.
The following sections provide further information regarding the cement train service, the train’s braking systems, the RCE (which was being used to operate the train and its braking systems), the design process and in-service history of the RCE, and other relevant subjects.
Context
Track and network
General information
The Western Line (which included the track section from Railton to Devonport) consisted of 160 km of 1,067 mm narrow gauge railway between Western Junction and Burnie, Tasmania. TasRail was the rail transport operator (RTO)[16] accredited as the rail infrastructure manager for this track section.
Devonport, located on the northern coast of Tasmania, had a ship loading facility. Devonport Yard was located near the facility at an elevation of 3 m above mean sea level (Figure 4).
Figure 4: Map of TasRail network
Source: Australasian Railways Association and TasRail, modified by the ATSB
Railton, located 21 km to the south of Devonport, was the location of a cement plant with a train loading facility. It was at an elevation of about 59 m.
The track descended from Railton towards Devonport with brief rising gradients encountered at 3 locations, the most substantial being in the vicinity of Horsehead Creek (Figure 4).
The safeworking system[17] on the Western Line was the advanced network train control system (ANCS) track warrant (see Network control).
Railton loading facility
Access from the Railton loading facility to the main line was through a set of trailable electric points where the main line and the track to the siding converged (Figure 1). On approach to the Railton cement siding with an empty cement train, a driver would (using the radio) remotely operate the points for entry into the siding. Once the train had entered the siding and was clear of the points, the points would automatically reset to the Western Line.
When departing Railton, the train would ‘trail’ through the points where the main line and track to the siding converged, which would automatically reset to the Western Line once the train was clear.
Onsite inspection by the ATSB at Railton on 22 September 2018 found no damage to infrastructure, including to the trailable electric points.
As previously noted, from the loading facility, the track had a downhill grade of 1 in 139[18] from the loading facility westward towards Devonport. There were no catch points or derailers fitted at the Railton cement siding (Figure 1). The purpose of catch points is to divert and derail an uncontrolled movement from a siding away from the main line. They are often installed on sidings with a downhill grade to the main line.
During March–April 2018, TasRail conducted a generic risk assessment for a runaway train across its entire network. An identified risk control was the provision of derailers or catch points at strategic locations based on a ‘…risk based approach of topography in yards to identify addition of catch points’ (that is, those with descending gradients to the main line).
TasRail advised that, at the time of the 21 September 2018 accident, Railton had not been risk assessed for consideration of catch point installation.
Track features
The maximum track speed between Devonport and Railton was 60 km/h, with lower speeds of between 40 km/h and 50 km/h at 3 locations associated with track curves.
Between Railton and Devonport, the line passed through 13 public level crossings, 10 of which were actively protected by flashing lights, which were designed to activate 20 seconds before the arrival of a train travelling at maximum track speed. The remainder were passive level crossings protected by stop signs only, relying on sighting distance. There were also some private level crossings. Table 1 shows the time and speed that the runaway train passed through the level crossings and other locations, together with the track speed at these locations.
Table 1: Location of runaway train and instances of excess speed (orange cells)
Track km
Time
Location
Track speed
(km/h)
Recorded speed (km/h)
109.700
0846:42
Cement siding – Railton
(Commencement of runaway)
Restricted[1]
0
109.900
0848:18
Cement Works Road level crossing (active)
Restricted
20
112.000
0851:41
Youngmans Road level crossing (passive)
60
51
116.100
0855:28
Dawsons Siding Road (active)
60 (exit 40)
76
118.000 (approx.)
0856:56
Fastest recorded speed
60
87.5
119.900
0858:18
Coal Hill Road (active) – Latrobe township
60
70
121.500
0859:48
Henslow Street (passive)
60
71
122.300
0900:22
Tarleton Road (active) – Tarleton locale
60
64
122.900
0900:54
Fosters Road (passive)
60
64
123.400
0901:23
Cornicks Road (active)
60 (enter 50)
65
125.100
0902:53
Sheffield Road (active) – Spreyton township
40
61
125.700
0903:29
Kelcey Tier Road (active)
40
56
126.500
0904:25
Stoney Rise Main Road (active)
40
54
126.900
0904:51
Durkins Road (active)
40
55
127.720
0905:13
Start of Horsehead Creek vicinity
40
60
128.520 (approx.)
0906:08
End of Horsehead Creek vicinity
40
43
128.870 (approx.)
0906:32
Slowest recorded speed
40
34
129.300
0907:47
Formby Road (active)
Restricted
47
129.380
0907:56
Devonport Yard entry (no. 88 points)
Restricted
48
129.440
0907:59
Bass Highway overpass
Restricted
49
129.530
0908:09
No. 85 points (adjacent fuel loading area)
Restricted
51
130.570
0909:06
Devonport cement silo
5
51
130.820
0909:29
Dead-end cement silos siding
(Train derailled)
Restricted
47
Restricted speed required the train to be regulated to a maximum of 25 km/h and able to stop within half the distance of line of sight.
As the train travelled through most of these crossings at higher than the maximum track speed, the warning time was reduced. In the worst cases, the actual warning times were about 10 seconds at Formby Road crossing and 14 seconds at Dawsons Siding Road crossing (see Figure 4).
Train drivers were also required to have the train’s headlight illuminated and sound the horn twice on approach to level crossings. In addition to the runaway train being uncontrolled, the locomotive’s headlight (which increased visibility of the train to the public) was extinguished and the locomotive’s warning horn could not be sounded.
On entering Devonport Yard, the train was diverted from the main line towards no. 2 road (silo siding) for unloading, via radio-operated points no. 88 and no. 85 (Figure 2). The Bass Highway from Launceston to Marrawah (in north-west Tasmania) crossed over the line on an overbridge just to the north of no. 88 points (Figure 5). A fuelling area was located just past no. 85 points (Figure 6).
Figure 5: Devonport Yard, no. 88 points
Diversion point from the main line (towards Burnie) via no. 88 points into Devonport crossing loop. The Bass Highway passed over the line shortly after the points. The speed limit for no. 88 points was ‘restricted speed’ (maximum 25 km/h). The runaway train was travelling at 48 km/h at this point. Source: TasRail, annotated by the ATSB
Figure 6: Devonport Yard, no. 85 points
Diversion point from the Devonport crossing loop via no. 85 points into no. 2 road (silo siding). A fuelling area was located on the left, shortly after the points. The speed limit for no. 85 points was ‘restricted speed’ (maximum 25 km/h). The runaway train was travelling at 51 km/h at this point. Source: TasRail, annotated by the ATSB
Network control centre
Overview
Network control services for the entire TasRail network were conducted from the network control centre located within the head office building in Launceston, Tasmania. It consisted of a primary workstation under the control of the shift network control officer (NCO), with a secondary adjacent workstation available if required for backup or supervisory purposes.
During business hours, other network control centre staff that were present included the network access manager, a relief NCO and roster clerk. However, outside business hours the NCO would be the only staff member in attendance.
Network control centre personnel
The NCO who was on duty at the time of the runaway, and answered the emergency phone call from the cement train driver, had 6 years experience, having joined TasRail as a trainee NCO.
The network access manager who liaised with emergency services during the accident had previously been a driver, operations manager and NCO. In 1999 they were involved as the planning manager during the introduction of remote control equipment (RCE) operations for the cement train service (see Operations with previous remote control equipment).
Network control
Safeworking on the TasRail network was administered through the Advanced Network train Control System (ANCS). This system involved the issuance of GPS-tracked electronic track warrants and authorities to trains and track users, using digital radio technology. ANCS would use GPS to monitor both track warrant authority exceedances and train overspeed, providing an alarm to the network control centre in the event of detected breaches.
Yards were under local control, with the ANCS system deactivated once a train was registered within yard limits. This meant that, in the event of a runaway from a yard, the train would be invisible to the ANCS system and no alarm would be registered in the network control centre by the ANCS. Rather, the NCO was reliant on a report of the runaway from either the driver or a member of the public.
Additional Selcall[19] alarms could be sent to the network control centre from the train (independently of the ANCS). Conditions that activated a Selcall alarm included:
emergency ‘cell call’ button on the TR class locomotive (this button provided the driver a direct means to activate the Selcall alarm)
RCE- or locomotive-induced vigilance penalty
RCE tilt function activation
any event that caused an emergency reduction of brake pipe pressure when the train was travelling over 3 km/h.
Alarms received by the network control centre were presented for actioning by the NCO on the communications screen to the right of the workstation. Phone and radio communications were also managed through this screen.
No Selcall alarms were triggered by train no. 604 during the runaway sequence. This was because the driver was not in the locomotive cab, the vigilance function and tilt function required communications between the RCE transmitter and receiver, and the train was stationary when emergency brake applications were applied.
In addition, the TR class locomotive was fitted with a system for automated locomotive (SAL) computer control, an electronics and automation system that included recording of input and output data. An aspect of this functionality was the provision of a remote connection to the locomotive that provided one-way viewing access to real-time information of digital and analogue locomotive inputs and outputs, including GPS location and speed. This is the system that the rolling stock access manager utilised to monitor the train’s position and speed during the runaway. Although access to this information (via the locomotive manufacturer’s online portal) was available to all TasRail staff, knowledge of this access was limited. For example, the NCO who was on duty at the time of the runaway was not aware that they had access and the network access manager advised they had never tried to access it.
Emergency procedures for runaway events
When an RCE-induced alarm, locomotive alarm or emergency call was received by the network control centre, NCOs were to follow the procedure in NA-PRO-800 (Emergency response). If an emergency was confirmed, the NCO was to immediately protect the area from other trains and call emergency services, advising of the incident and any secondary hazards (for example, dangerous goods).
TasRail procedure SHE-PRO-201 (Administering the incident and emergency response procedure) required the NCO to conduct an evaluation of the potential magnitude of the event using the ‘TasRail 6 x 6 risk assessment matrix’. This would inform the internal TasRail response (for example, appointment of an incident site controller, organising drug and alcohol testing, and reporting of the incident to external authorities). All incidents were to be recorded in the ‘risk wizard’ computer system for tracking and, where required, follow up.
The emergency response procedures did not contain any additional instructions for managing incidents that may have required a time-critical tailored response (for example, a runaway or track warrant authority exceedance).
Driver information
The driver of train no. 604 on 21 September 2018 had worked for TasRail since the early 1970s in various roles including as a driver, based at Devonport for almost all of this time. The cement train was exclusively operated with remote control equipment (RCE) since its introduction in 1999.
The driver was assessed competent as a train driver, with Australian Qualifications Framework qualification TLI42615 (Certificate IV in train driving) awarded to the driver in March 2017. The last reassessment of the driver’s competence for ‘safety critical competency units’ of this qualification was conducted on 9 March 2018, with the driver deemed competent.
In relation to operation of the RCE, the driver was assessed as competent in the unit of competency TLIC3082A (Operate a locomotive by portable remote control) in February 2015. The last reassessment of the driver’s competence for TLIC3082A was conducted on 22 February 2017 (using generation 2 RCE), with the driver deemed competent. Assessed observations included direction change, interpreting communication failed mode and control fault mode, resetting a communication failed mode occurrence, commanding and resetting an emergency stop application, and loading procedures at Railton.
There was no indication that the driver was in any way impaired at the time of the runaway accident. The driver reported they had been sleeping well prior to the shift and they did not report any issues with fatigue. They were about 7 hours into their work shift at the time of the accident. A drug and alcohol test was administered after the accident and returned a negative result (that is, no alcohol or drugs was detected).
Train information
General information
Cement train no. 604 was 220 m long and weighed about 1,132 t. It consisted of:
a TR class locomotive (TR11) at one end
16 cement powder wagons (14 of which were loaded at the time of the accident)
a driver’s van wagon (DV1) at the opposite end.
The cement train was a regular bulk freight service that operated 24 hours per day, 7 days per week, completing about 6 service cycles between Railton to Devonport and return each day. TasRail was the rolling stock operator, and the cement train service was considered one of its most important, hauling more than 1 million tonnes of cement powder annually.
The cement train was operated as a driver only operation,[20] with motive power provided from the TR class locomotive, which was always positioned at the Devonport end of the train. This locomotive was always controlled by a driver using RCE:
from Railton to Devonport, with the driver located in the cabin of the locomotive (the locomotive was hauling the consist as the leading vehicle)
from Devonport to Railton, with the driver located in the cabin of the driver’s van (the locomotive was propelling the consist as the trailing vehicle)
with the driver outside of the train in the loading facility at Railton, while loading cement powder
with the driver outside of the train in the unloading facility at Devonport, while unloading cement powder.
The loading and unloading operations took longer than the transits; therefore, the driver was outside of the train for the majority of an operational cycle.
The driver’s van was a reclaimed Y class locomotive with no engine or traction motors. It was solely used for the driver to observe the track ahead while controlling a locomotive at the rear of the train using the RCE.
On a non-remotely-controlled train, the leading locomotive in the direction of travel has sole control authority over the train’s airbrake systems. To enable this, the lead locomotive’s braking system is conditioned to ‘lead cut-in’ mode. This controls the application and release commands of the braking systems. Trailing locomotives (under command of the lead locomotive) are conditioned to ‘trail cut-out’ mode, their response being similar to that of a trailing wagon.
For TasRail’s remotely-controlled trains, the locomotive was conditioned to the trail cut-out mode, and so was essentially the equivalent to a trailing locomotive. The RCE took the place of a ‘lead’ locomotive, giving all electrical and airbrake commands to the physical ‘trailing’ locomotive.
Further information about the braking systems on the train is provided in Train braking systems.
The RCE on train no. 604 was the generation 3 RCE developed and manufactured by Air Digital Engineering (ADE) for TasRail. Further information about the RCE is provided in Remote control equipment.
Post-accident inspection
Onsite inspection of the accident scene in Devonport by the ATSB on 21 September 2018 found that the TR class locomotive and first 8 cement wagons were in a derailed state. The continuity of the brake pipe, operational status of the triple valve braking equipment (see Automatic brake) and condition of brake blocks on the rolling stock was examined, where possible.
It was observed that the RCE receiver was correctly connected to the TR class locomotive, with the locomotive brake pipe continuous to the first wagon. Derailment damage prevented verification of brake pipe continuity across wagons 1 to 5 and across wagons 6 to 7. Likewise, the operational status of the triple valve on wagon 4 and brake block condition on wagons 1–7 were unable to be verified due to accident damage. The status of all other brake pipe continuity points and triple valves on the train was found to be correct, and brake blocks present and within wear tolerances.
When the ATSB inspected the TR locomotive, it was shut down with the batteries turned off and with all circuit breakers in the ‘off’ position. However, TasRail advised the ATSB that these actions were performed as part of the emergency shutdown after the derailment and that all circuit breakers were correctly configured for operation when its personnel arrived at the site. All other cab switches and equipment were observed to be generally conditioned correctly for RCE operation. The following was also noted:
the locomotive’s automatic brake handle was in ‘release’ (no effect as in ‘trail cut-out’ mode)[21]
the locomotive’s independent brake handle was in ‘release’ (no effect as in ‘trail cut-out’ mode).
In summary, although anomalies were noted, the ATSB did not identify any locomotive configuration issue that would have caused the train not to respond to RCE commands from the driver. In addition, as previously discussed (see also Appendix A), analysis of available evidence identified that the locomotive, including its control and safety systems, operated as designed.
Operational inspections and maintenance
There was one train consist dedicated to the cement service between Devonport and Railton. The TR class locomotive was swapped every 30 days for servicing.
The cement wagons were inspected every 4 months at the East Tamar Junction wagon maintenance facility. Between these inspections, the driver of the cement service would perform a general inspection for obvious mechanical wear and damage. One side would be checked at Devonport after unloading and the other side at Railton prior to loading.
The driver’s van was inspected at 2-monthly intervals at either East Tamar Junction or Burnie maintenance facility, with a general inspection undertaken in-between as with the cement wagons. Basic safety checks occurred every 7 days.
In-service tests of the train braking systems (noted on the brake test certificate) were valid for 7 days unless the train was idle for more than 4 hours or a shunt of the rolling stock had occurred.[22] Records showed that the last shunt of the consist had occurred on 10 September 2018, with the last terminal examination (including all relevant brake tests and a test of the RCE tilt[23] and vigilance[24] functions) conducted the day before the accident.
During its 30-day operational period, the locomotive was given a basic safety and operational check every 7 days, which was to include (among other things) provision of consumables, multiple unit functionality, sand, direction, traction power and brake tests. The record of the basic safety and operational check, completed for TR11 the night before the accident, did not mark off the multiple unit functionality, sand, direction, traction power and brake checks. A review of TR11’s previous basic safety and operational checks found these items were not always recorded. TasRail reported that, as the train was operated almost continually, functions such as traction power and braking were effectively being checked on an ongoing basis. As previously noted, no problems with the serviceability of the locomotive were considered to be involved in the development of the runaway sequence.
Propelling operations
A unique aspect of TasRail’s cement train operation was the use of a single motive power source, the TR class locomotive, at the rear of the consist (in a propelling configuration). This occurred when the train was operated in the empty direction, from Devonport to Railton, with the locomotive propelling (pushing) the train at track speed on the main line. This generated increased buff (compressive) forces and vertical loads in the couplings between wagons, particularly between the locomotive and first few wagons compared to a train that was being pulled by a locomotive.
Propelling was not a factor in the runaway accident sequence. However, as part of its investigation, the ATSB considered how RCE was integrated into the rail operation. TasRail’s use in a propelling configuration on a main line from Devonport to Railton was permitted without restriction while being controlled from the driver’s van. As far as could be determined, no other train service in Australia used a similar rear-propelling-only configuration.
Due to the increase in lateral force against the rail head and reduction in wheel load, excessive buff forces can be a cause of wheel lift and flange climb or jack-knife derailments.[25] Careful and gradual increase of buff forces by the driver is therefore required during propelling movements to minimise the chance of derailment. Although these effects are greatest with loaded wagons, they remain present with empty wagons.[26]
TasRail provided some train handling instruction to drivers, with emphasis on efficiencies of train movement (for example, reduction in fuel usage and brake block wear). It did not provide instructions related to the rear-propelling configuration used with its RCE, including at maximum track speed.
Further aspects of the use of the RCE with propelling operations are discussed in Reduced effectiveness of automatic emergency brake.
Train braking systems
General description
On non-remotely-controlled (or conventional) TasRail trains, a driver operated the controls from within a locomotive cabin at the lead of the train consist. With these controls, a driver could issue commands for tractive effort to move the train and braking effort to slow and stop the train. In addition, several safety features were available to the driver, such as the vigilance system and the emergency engine stop function (which shut off the diesel engine).
This section provides an overview of the braking systems on the TR class locomotive and how they are applied on a non-remotely-controlled train. The RCE, and the means of using it to control braking, is discussed in the next section.
There were 3 braking control systems fitted to the TR class locomotive:[27]
automatic brake (a type of airbrake that controlled brakes on the entire train)
independent brake (a type of airbrake that controlled brakes on the locomotive only)
electrical dynamic brake on the locomotive.
The use of each of the 3 braking systems depended on the circumstance during which braking effort was required. For both airbrake systems, braking was achieved by the air-powered application of brake blocks[28] directly to locomotive or wagon wheels.
The 2 airbrakes (automatic and independent) were electronically-controlled and air-actuated on the leading locomotive. The airbrake system consisted of a driver’s cab handle unit (CHU, Figure 7), which sent electronic commands to the pneumatic operating unit (POU) located in the locomotive engine room. The POU then controlled the pneumatic responses to slow or stop the train.
Figure 7: Cab handle unit in TR locomotive
Source: TasRail, modified by the ATSB
Compressed air for the airbrake system was provided by a compressor and stored in 2 main reservoirs, both of which were on the locomotive:
Main reservoir no. 1 (MR1) air was supplied to other rolling stock by means of the main reservoir equalising pipe (MREQ).
Main reservoir no. 2 (MR2) was supplied air pressure from MR1 via a one-way check valve. MR2 air was used by the POU to effect braking commands.
Automatic brake
Introduction
The automatic brake applied and released brakes on both the hauling locomotive(s) and any trailing rolling stock (that is, across the entire train). It was termed the ‘automatic’ brake because the brakes automatically applied when air pressure was lost, such as in a major derailment or train separation.[29] In normal operation, the automatic brake was used as the main mechanism for slowing and stopping the train. The driver did this using the CHU automatic brake handle.
Integral to the function of the automatic brake was a continuous pipe of air that ran from the front to the rear of the train, known as the ‘brake pipe’. Flexible rubber hoses at each end of a wagon allowed connection of the brake pipe with a locomotive or other wagons.
In simplified terms, air pressure in the brake pipe performed 2 functions:
acted as a signal to control brake application and release on both the locomotive(s) and the rest of the train
provided a conduit for ‘charged’ (pressurised) air from the locomotive(s) to storage reservoirs in the wagons, which then provided the braking application force.
Decreases in brake pipe pressure applied proportional increases in pressure in locomotive and wagon brake cylinders, as long as there was enough air in the storage reservoirs to make the application. Increases in brake pipe pressure, above a small amount, fully released the brakes.
Means of operation
The lead locomotive charged the brake pipe with air from MR2 to an air pressure of 500 kPa when brakes were fully released. In addition to the brake pipe, each cement wagon[30] was fitted with a pneumatic control valve (‘triple valve’), auxiliary (storage) reservoir, and brake cylinders.[31] The control valve responded to changes in pressure within the brake pipe. On sensing a reduction in brake pipe air pressure, the control valve would transfer air from the auxiliary reservoir to the brake cylinder, which would apply brakes. Figure 8 shows how this system worked in practice for a brake application.
Figure 8: Wagon airbrake application process
The image is a simplified representation of an airbrake application on a wagon. The control valve consisted of rubber diaphragms, springs and ports that responded to reductions in brake pipe air pressure to perform the required brake apply response. Source: The Railway Technical Website, modified by the ATSB
A reduction in brake pipe pressure would similarly apply the brakes on the hauling locomotive. The locomotive’s POU would read the brake pipe pressure and command a corresponding brake cylinder pressure of up to a maximum of 350 kPa on the locomotive.
Once the required speed reduction had taken effect, the driver would place the CHU automatic brake handle in release, which restored the brake pipe pressure to 500 kPa (Figure 9). On sensing an increase in brake pipe air pressure, the control valve would vent air from the brake cylinder to release the brakes, and also allow air from the brake pipe to recharge the auxiliary reservoir to 500 kPa.
Figure 9: Wagon airbrake release process
The image is a simplified representation of an airbrake release on a wagon. The control valve consisted of rubber diaphragms, springs and ports that responded to an increase in brake pipe air pressure to perform the required brake release response. Source: The Railway Technical Website, modified by the ATSB
Driver controls
The CHU automatic brake handle had 6 detent positions for driver control (Figure 7):
release (REL) – brake pipe charged to 500 kPa
minimum service (MIN) – brake pipe reduced to 450 kPa at a service rate
full service (FULL) – brake pipe reduced to 350 kPa at a service rate
suppression (SUP) – brake pipe reduced to 350 kPa at a service rate (for penalty brake reset)
continuous service / handle off (HO) – brake pipe reduced to 0 kPa at a service rate
emergency (EM) – brake pipe reduced to 0 kPa at a rapid rate.
The operational area between minimum service and handle off detents reduced the brake pipe pressure at a steady service rate. The emergency position vented brake pipe pressure at a rapid rate. Maximum braking effort occurred when brake pipe pressure was reduced to 350 kPa, known as a full-service brake application. At this point, the brake pipe, auxiliary reservoir and brake cylinder pressures all equalised with each other at about 350 kPa. The ATSB was advised by the airbrake manufacturer that a 350 kPa brake cylinder pressure on both wagons and locomotives was unlikely to result in wheel lock.[32]
Although the driver could graduate an increase in automatic brake effort by reducing brake pipe pressure incrementally between minimum-service and full-service positions, it was not possible to graduate a release. Once the control valve on the cement wagons sensed a pressure rise over 12.5 kPa in the brake pipe, it would release the brakes.
Brake pipe charging flow indicator
During an automatic brake release, MR2 air supply, which was fed from MR1, would recharge the brake pipe. The brake pipe charging flow indicator provided an indication of the flow rate from MR2 to the brake pipe. It was displayed to the driver on the TR class locomotive in-cab display screen. The purpose of the brake pipe charging flow indicator was to indicate:
when the brake pipe (and thereby wagon auxiliary reservoirs) was fully charged. This avoided the circumstance of ‘running out of air’, whereby repeated brake applications with short release (brake pipe recharge) periods could result in loss of auxiliary reservoir air pressure. As a result, no air pressure would be available for the wagon brake cylinders.
when tractive power could be applied from a standing start (that is, once the train’s brakes were fully released). This avoided high locomotive tractive effort against the train’s brakes causing high draft forces, resulting in possible coupling or draft gear[33] damage, train separation or rail burn.[34]
uncommanded loss of brake pipe pressure. This could indicate a broken brake pipe hose, derailment or train separation event; all of which required an immediate driver response.
Indicators of main reservoir pressure were also available, and a driver could monitor for changes in these pressures for the same purposes.
Penalty conditions
In addition to the handle off position on the CHU automatic brake handle, there were a number of penalty conditions for a locomotive set to ‘lead cut-in’ mode that would result in a brake pipe pressure vent to 0 kPa (brake application) at a service rate. These included:
Vigilance penalty: a penalty mode triggered by the vigilance system on the locomotive. The vigilance system was a protection against driver incapacitation. A penalty occurred if a driver did not acknowledge an audible warning light and alarm or make a control input within a periodic 15-second invitation period. This system was active when the locomotive was in lead cut-in mode and brake cylinder pressure was <170 kPa (that is, likely moving).
Overspeed penalty: a penalty mode triggered by the detection of locomotive overspeed. The TasRail TR class locomotives were configured so that an overspeed penalty occurred if the locomotive speed reached or exceeded 88 km/h. The maximum speed on the TasRail network was 70 km/h.
CHU to POU communications loss
braking system power supply failure.
When the brake pipe pressure fell below 297 kPa, the locomotive would operate the power control switch (PCS). This would remove the locomotive’s generator excitation, preventing further tractive power and dynamic brake from being generated. The driver reset the penalty condition by clearing the cause of the penalty and placing the CHU automatic brake handle in suppression, before returning it to release. The PCS would reset once the brake pipe pressure reached over 428 kPa.
Emergency brake
The driver could activate emergency braking by either the CHU automatic brake handle or by activating a separate emergency dump valve in the locomotive cabin. In addition, the brakes would automatically apply across the train in the event of a brake pipe rupture, such as during a derailment or train separation. In both cases, braking would be initiated by venting the brake pipe to 0 kPa.
On sensing a rapid (emergency) brake pipe pressure drop (that is, more than 69 kPa / second, either commanded by the driver or initiated by brake pipe rupture), the hauling locomotive POUs would energise the emergency magnet valve (EMV), increasing the rate of pressure drop. In the case of a leading locomotive set to lead cut-in mode, the POU would also prevent recharge of the brake pipe. The hauling locomotive would enact a PCS request and emergency sanding,[35] the latter if the train speed was over 3 km/h. As no further braking effort was achieved once the brake pipe reduced below 350 kPa (due to pressure equalisations), emergency braking only increased the rate at which brake application occurred. Emergency braking did not increase the level of braking effort.
For a locomotive configured to lead cut-in mode, a reset of an emergency brake application by the driver (after clearing the cause) required the CHU automatic brake handle to be placed in the emergency position for a minimum of 60 seconds. The purpose of this timeout was to ensure the train had come to a complete stop before a brake release. This prevented train separation from coupler breakage due to heavy braking at the rear of the train coinciding with releasing brakes at the front. On trailing locomotives configured to ‘trail cut-out’ mode, the POU’s EMV would self-reset after 20 seconds in preparation for a brake pipe recharge command from the lead locomotive.
Emergency vent valves
The TR class locomotive was fitted with 2 emergency vent valves, called ‘VX vent valves’ by the manufacturer. They were a pneumatic addition to the brake pipe at each end of the locomotive (see inset Figure 11), which was a requirement in the Australian Standard for locomotive braking systems.[36]
On sensing a rapid (emergency rate) reduction of brake pipe pressure, the VX vent valves released brake pipe air pressure, increasing the speed of brake pipe pressure drop in the event of an emergency. After the brake pipe pressure reduced sufficiently, the valve would internally equalise and reseat (stop venting). As VX vent valve operation was related to internal pressure equalisation, if air was still being supplied to the brake pipe when equalisation occurred, the VX vent valves would reseat, stop venting, and permit recharge at pressures above 0 kPa.
Vent valves were not fitted to the cement wagons or the driver’s van. Prior to using TR class locomotives, TasRail used DQ class locomotives. These also were not fitted with vent valves.
In summary, both the locomotive’s VX vent valves and EMV were designed to assist the rapid exhaust of the brake pipe on detection of an emergency brake command, either by the driver or through brake pipe rupture. Additionally, this ensured activation of the locomotive’s PCS, thereby removing traction power. Timeouts for EMV reset applied to prevent early emergency brake release and possible train separation.
Independent brake
The independent brake applied and released brakes on hauling locomotives only, independently of any trailing rolling stock. The driver controlled this using the CHU independent brake handle to adjust pressure in a ‘control pipe’. The braking could be graduated between the release (REL) and apply positions, the latter resulting in an air pressure charge (effected by the POU) of up to 350 kPa in the control pipe.
Flexible rubber hoses on each end of the locomotive were provided to allow connection of the control pipe with other locomotives. This provided a facility for one or more trailing locomotives (if used) to receive the control pipe pressure from the lead locomotive to effect independent brake applications on the trailing locomotive(s).
Dynamic brake
On the TR class locomotive,traction motors could also be used to slow (but not stop) the train using the dynamic brake system. This reduced wear on braking components and reduced the risk of having insufficient air pressure to apply brakes after repeated applications and releases.
To engage the dynamic brake, the driver would return the throttle / dynamic brake handle to idle from the traction power zone then move it into the dynamic brake zone (Figure 10). This action would change the traction motor fields from a tractive power to a generator configuration. The subsequent power generated by the moving locomotive wheels was fed to resistor grids and dissipated as heat, resulting in a retarding force.
Figure 10: Driver’s throttle (traction power) and dynamic brake controller
Source: TasRail, annotated by the ATSB
As previously discussed (see also Appendix A), analysis of available evidence identified that during the accident sequence the RCE initiated a momentary emergency vent of the brake pipe, which resulted in the locomotive’s VX vent valves and EMV applying to exhaust the brake pipe further. The RCE also triggered 2 emergency brake applications and releases, which did not replicate any fault condition behaviour documented for the RCE. The locomotive electronic airbrake system operated as designed.
Remote control equipment
Background
In about 1999, TasRail’s predecessor, ATN Tasrail, altered the business operating model for its existing cement train service, changing from a 2-person crew configuration to a driver only operation configuration using RCE. In addition to reducing staffing costs and exposing fewer staff to hazards during loading and unloading, this operating model allowed for the use of a single locomotive, without the requirement to either run-around or turn the consist at Devonport and Railton.
ATN Tasrail found during the tendering process that there was limited market availability of portable main-line RCE units.[37] Shortly afterwards, ADE proactively approached ATN Tasrail with its developmental RCE system. Based on the outcome of trials, in August 1999, the generation 1 RCE system was purchased by ATN Tasrail, being ADE’s first RCE customer.
In about August 2010, the operator (now TasRail) replaced generation 1 RCE with ADE’s generation 2 RCE. In turn, following issues with the ongoing reliability of generation 2 RCE and other factors, TasRail commissioned ADE to develop the generation 3 RCE, and it was introduced in February 2018 (see Remote control equipment development). Generation 3 RCE was fitted to train no. 604, and unless otherwise noted, references to RCE in this report relate to the generation 3 RCE.
ADE advised the ATSB that, in addition to TasRail, 5 Australian rail transport operators (RTOs) used the generation 2 RCE. However, all had ceased to do so by the time of the accident in September 2018. TasRail was the only RTO that used the generation 3 RCE.
TasRail primarily used the RCE for cement train services on the section of railway between Devonport and Railton. In addition, it permitted RCE operations for:
emergency banking[38] purposes (assisting with heavy loads) over Don Hill between Devonport and Leith
transfer of a locomotive and driver’s van to Burnie for maintenance purposes
transfer of cement wagons to East Tamar Junction for maintenance purposes.
System overview
The RCE comprised 3 main components:
a transmitter, a portable unit carried by the driver
a receiver, mounted on the locomotive, which interfaced with the locomotive’s electrical systems via a multiple unit cable
an air box, mounted on the locomotive, which interfaced with the locomotive’s airbrake system via air hoses.
The RCE receiver received commands from the RCE transmitter, with communications between the 2 components linked on start-up of the RCE system. When operated by the driver, the transmitter sent remote control signals via radio to the receiver. The receiver then sent electrical signals to the air box to enact airbrake commands, or to the locomotive for throttle, direction, and other commands.
The RCE replicated most of the controls available on the locomotive, including automatic brake, independent brake, dynamic brake, engine, and direction controls. An emergency stop switch was intended to imitate the response to placing the locomotive’s automatic brake handle in the emergency position (see Emergency stop function).
Interface with the locomotive
Generation 2 RCE’s receiver and air box were located at the front of the previously-used DQ class locomotives. On introduction of the TR class locomotives, the location was changed to the rear of the locomotive, due to the absence of a footplate or handrails to allow mounting of the receiver and air box at the front of the locomotive. The rear location was continued with generation 3 RCE. In this arrangement, the receiver was placed on the rear locomotive footplate with the air box located on the right-side footplate (Figure 11).
Figure 11: Rear of TR class locomotive, showing interface with generation 3 RCE
Image taken at the accident site after removal of the wagons. Note: the brake pipe tap which connected the air supply to the train has been closed during the wagon removal process. Operation of the VX vent valve is discussed further in Train braking systems. Source: ATSB
The air box location affected some aspects of the braking system response, due to it being connected to the airbrakes at a different point in the pneumatic circuit (see Remote control equipment emergency brake reset).
A multiple unit electrical cable from the RCE receiver was plugged into the TR class locomotive’s multiple unit socket to provide the receiver with a 74 V DC power supply. The multiple unit cable also carried electrical commands (for example, direction, throttle and dynamic brake settings, sand and horn) from the receiver to the locomotive.
A communication cable from the receiver to the air box carried electrical airbrake commands for the pneumatic system within the air box to enact. The air box was supplied with pressurised air from main reservoir no. 1 (MR1) by the main reservoir equalising pipe (MREQ). To allow this, the driver was required to condition the TR class locomotive’s braking system to ‘trail cut-out’ mode. This gave control of the airbrake to the RCE, which could directly command:
application of the train’s automatic brake through brake pipe pressure reduction via a brake pipe valve and/or a separate emergency brake valve in the RCE air box
release of the train’s automatic brake by recharging the brake pipe with MREQ air (not MR2 air as on a TR class locomotive without RCE)
release of the locomotive’s brake cylinder pressure during an automatic brake application through charging the independent release pipe (‘bail-off’ function)[39]
application of the locomotive’s independent brake through charging the control pipe with MREQ air (not MR2 air as on the TR class locomotive without RCE)
release of the locomotive’s independent brake through exhausting the control pipe
penalty brake applications.
Figure 12 shows the interface between the locomotive and the first wagon. The brake pipe connection shown allowed reductions and increases in brake pipe pressure by the RCE, enacted through the adjacent brake pipe connection (Figure 11), to be registered by the wagons and driver’s van. The MREQ air hose connected to the first wagon provided MR1 air supply to the wagon roof hatches, which were operated for cement powder loading purposes at Railton.
Figure 12: Interface of RCE from rear of TR class locomotive to first wagon
Image taken on the day of the accident. Source: ATSB
Driver operation
The RCE transmitter (Figure 13) was designed to replicate several controls available to the driver as if they were operating a non-remotely-controlled train consist from the cabin of a locomotive. These were:
generator field switch (by selecting a throttle notch while in forward or reverse)
engine run switch (by entering ‘run’ mode).
Figure 13: Generation 3 RCE transmitter
Image shows the RCE transmitter as initially supplied to TasRail for generation 3 RCE. Differences to generation 2 RCE’s transmitter included reversal of the emergency stop position, and the addition of the dynamic brake switch. The transmitter’s key switch was located on the left side of the transmitter (not shown). Source: Air Digital Engineering, annotated by the ATSB
The direction controller had 3 positions:
forward (up)
neutral (middle)
reverse (down).
The neutral position was gated; that is, the driver had to lift the selector switch up to go past the neutral position. Although not recommended by ADE in its RCE operation manual, it was possible to immediately change direction from forward to reverse and vice versa on the transmitter without pausing in the gated neutral position, if the selector switch was lifted clear of the gate. In contrast, on a locomotive the change of direction was delayed through the length of travel of the direction controller between the forward and reverse positions and a detent for the neutral position.
The RCE transmitter’s throttle and brake controls had a significantly shorter travel and lighter feel than those of the locomotive’s throttle / dynamic brake handle, automatic brake handle and independent brake handle. Detents for these controls were provided on the transmitter to enable the driver to feel the lever position more precisely.
In summary, although most commands available to a driver when operating from the TR class locomotive cabin were replicated on the transmitter, the control interface was significantly different.
The RCE transmitter display screen was a liquid crystal display (LCD). The display screen provided information to the driver of:
direction selected
throttle / dynamic brake notch setting
brake pipe and control pipe pressures
the RCE’s operating mode (for both the transmitter and receiver), including fault states.
The transmitter’s key switch was located on the left side of the transmitter. Its positions included:
on (used during normal train operation)
off
interlock (used during loading and unloading, see below).
The RCE transmitter was attached to a vest worn by the driver when operating from the locomotive or outside the train, and placed in a receptacle when the RCE was being operated from the driver’s van.
Train loading process
After the train entered the Railton loading facility, the driver would stop the train and alight from the driver’s van at one end of the silos and, using the RCE transmitter, operate the train slowly past the driver, performing a general inspection and checking the bottom discharge doors were closed on the Western Line side of the train. The driver would stop the train once the inspection was complete, aligning wagons 1 and 2 under the silos in preparation for loading.
The driver would then enter the loading facility and proceed to the loading platform (Figure 14). Once there, they would:
configure the RCE transmitter with direction in neutral, throttle idle, independent brake fully applied, and automatic brake in at least the ‘initial’ position
switch the transmitter key to the ‘interlock’ position (activating the Railton loading interlock, which prevented train movement with the loading chutes in the lowered position)
remove the key from the transmitter
insert the key into the loading control panel, located on the platform, to allow loading operations.
Figure 14: Railton loading facility
Source: ATSB
Selecting ‘interlock’ allowed the transmitter key to be removed from the receiver, and also caused a minimum-service brake application (50 kPa brake pipe pressure reduction) to be applied and held on the train (see Automatic brake). Any manipulation of the transmitter’s controls while in ‘interlock’ mode (apart from increasing an automatic brake application) resulted in an emergency brake application.
In the event of an overshoot of the stopping mark, the train would be reversed and realigned. Due to the weight of a heavily-loaded train combined with the descending grade, if an overshoot occurred towards the end of the train loading process, it would sometimes be necessary to release the 50 kPa automatic brake application to allow re-positioning.
Once train loading was complete, the driver would walk from the loading facility to the TR class locomotive at the Devonport end of the Railton cement siding, board the locomotive and drive the train to Devonport using the RCE transmitter.
Safety design elements
Overview of safety features
The safety features in the generation 3 RCE’s design included, among others:
emergency stop function
vigilance function
processor watchdog and self-reboot
communication failed mode
control fault mode
failed train brake application function
dual direction fault interlock.
In addition, the system was designed so that many faults involving the emergency stop or braking signal paths would result in venting the relevant pipe(s) to apply brakes. For example, relays and pneumatic control valves had to work correctly in order for the brake pipe not to be vented.
Emergency stop function
The RCE used electrically-controlled pneumatic valves within its air box to vary air pressure within the train’s braking system. When activated, the emergency stop mode applied emergency braking by the removal of an electrical signal to the emergency brake valve in the RCE air box. This released air at a rapid rate from the train’s brake pipe.
The emergency stop mode would normally be entered in normal operation (that is, after initialisation) when:
the driver activated the RCE transmitter’s emergency stop switch (see Figure 13)
the driver commanded traction with excessively low brake pipe pressure (less than 300 kPa)
software commanded a brake pipe pressure below or equal to 450 kPa, and then sensed brake pipe pressure stayed above 480 kPa for more than 15 seconds (that is, when an attempted brake application failed); see Failed train brake application function.
In the emergency stop mode, the RCE receiver software would issue commands to:
open the RCE emergency brake valve to reduce brake pipe pressure rapidly to 0 kPa (which would apply locomotive and wagon brakes)
set control pipe pressure to 325 kPa (which would separately apply locomotive brakes)
command the release of sand for rail-wheel friction
command the locomotive to remove traction and set the direction to neutral.
The RCE required a functioning radio link to respond to a driver-commanded emergency stop application. ADE considered a driver’s responses to failures during the design process. It emphasised that a driver would always have the availability of the emergency stop function, provided there was a radio link (see also Safety integrity).
The RCE receiver could not be triggered into the emergency stop mode with a rapid drop of brake pipe pressure (for example, as a result of brake pipe rupture during a derailment or train separation). If this occurred, the receiver would attempt to regain the last-commanded brake pipe pressure. The consequences of this are discussed further in Reduced effectiveness of automatic emergency brake.
Some hardware faults such as power failure would also result in a hardware-induced emergency brake application: the train’s brake pipe would be rapidly vented in a similar manner to a manual activation of the emergency stop mode by the driver. The locomotive reacted to this in the same way as it did to a brake pipe rupture (as described in Emergency brake), sensing the resulting reduction in brake pipe pressure and acting to create its own brake pipe pressure exhaust. If the pressure in the brake pipe was low enough, the locomotive also would remove propulsion.
Vigilance function
The purpose of the RCE’s vigilance function was to stop the train in the event of driver unresponsiveness (for example, due to incapacitation). On a non-remotely-controlled train, this function was controlled by the TR class locomotive. It required the driver to either acknowledge a periodic vigilance warning or make a control input to avoid entry into a penalty mode, which would result in removal of traction power and a penalty brake[40] application (called a vigilance penalty).
The RCE vigilance function required the driver to acknowledge a vigilance warning on the RCE transmitter at set time periods and it also had a penalty mode that resulted in a loss of traction power and a penalty brake application. However, the ATSB found that:
the RCE vigilance function required a radio link to be maintained between the transmitter and receiver for it to operate
the penalty mode, and the resulting penalty brake command and alarm to the network control centre, were inoperative when the RCE was in the communication failed mode
the penalty mode could be reset after 30 seconds, unlike the 60-second timeout of the TR class locomotive’s vigilance function (shorter reset times can result in increased in-train forces due to the time taken for braking signals to traverse the entire train).
During remotely-controlled train operations, the TR class locomotive vigilance control system was suppressed as a necessary consequence of the locomotive having to be in the ‘trail cut-out’ mode to accept commands from the RCE (as previously discussed under Train information).
Processor watchdog and self-reboot
The RCE receiver included a ‘watchdog’ function. The software regularly toggled one of the processor’s outputs, which was monitored by an electronic circuit. In the event of a major processor hardware failure or fatal[41] software error, the unchanging signal would cause the watchdog circuit to remove power to (and thereby open) the emergency brake valve in the RCE air box, depleting air from the train’s brake pipe and activating the emergency brake without entering the emergency stop mode. That is, the watchdog function was separate to the emergency stop function.
Unlike the emergency stop function, the watchdog function did not directly affect control pipe pressure, command the release of sand, or change the direction. Rather, the circuit would cause the processor to reboot and reinitialise the RCE receiver. The driver could then proceed by completing steps similar to the normal start-up sequence.
The receiver was not able to operate until the watchdog was activated, which happened early in the initialisation sequence. The watchdog was triggered using software timers separate from the main processing loop, so it was unable to detect ‘endless loops.’ Further, watchdogs are unable to detect other non-fatal software problems (that is, ‘bugs’ where the software continues running but does not operate as intended).
Communication failed mode
In instances where communications between the RCE transmitter and receiver were interrupted for more than 4 seconds, the receiver would enter the communication failed mode. When activated, this mode would:
remove traction power
reduce the brake pipe pressure to 400 kPa (an intermediate application of automatic and independent brakes)
if the brake pipe pressure was already less than 375 kPa, set the control pipe pressure to 325 kPa; otherwise, retain the previous control pipe pressure setting
set the direction to neutral after 10 seconds
display ‘COMF’ on the transmitter display to indicate that the system was in this mode.
If communications resumed while in this mode, the receiver would not respond to any command messages from the driver (from the transmitter) other than the tilt function, emergency stop function activation and the reset command (see Failure mode reset procedures). These driver commands still relied on a radio message from the transmitter being received and processed by the receiver to function.
Control fault mode
In certain circumstances where the locomotive response did not replicate the driver’s transmitter command, the receiver would enter the control fault mode. These conditions were:
a throttle command mismatch
the generator field remaining on when commanded to be off
failure of the independent brake to apply.
In response, if the train was moving under traction power with brakes released, the receiver would, on initiation of the control fault mode, remove traction power. Then, after 10 seconds:
set the direction to neutral
reduce the brake pipe pressure to 350 kPa (equivalent to a full-service brake application)
set the control pipe pressure to 325 kPa (maximum locomotive brake cylinder pressure)
display ‘CNF’ on the transmitter display to indicate that the system was in this mode.
The RCE was designed to accept and respond to emergency stop commands from the driver while in the control fault mode.
Failed train brake application function
The RCE had a function that was described in the RCE operation manual as ‘Failed Train Brake Application results with an Emergency Mode application of the system’. The system state required to activate this function was not described, nor was a procedure for diagnosing and correcting the condition.
ADE advised the ATSB that the fault mode resulted from the brake pipe pressure being at or more than 480 kPa for 15 seconds while an automatic brake command was in effect. That is, it resulted from a failure of either the RCE receiver or the air box to effect an automatic brake application requested from the transmitter. This was conceptually similar to the control fault mode but applied to the automatic brake. The RCE would then enter the emergency stop mode.
Dual direction fault interlock
As part of the generation 2 RCE upgrade, ADE introduced a dual direction fault interlock. The interlock created an emergency brake application by the RCE receiver (through hardware) if both the forward and reverse direction outputs were selected simultaneously, as this was believed to have occurred during a February 2010 accident (see Operations with previous remote control equipment).
The dual direction fault interlock was retained with the introduction of generation 3 RCE. Although present in both generation 2 and 3 RCE, the dual direction fault interlock was not described in the RCE operation manual for either system.
Independent stop mechanisms
When using the generation 1 RCE, TasRail fitted an independent remote stop mechanism external to the RCE to 4 DQ class locomotives. A button on a portable hand-held radio carried by the driver would send a radio signal to a receiver (separate to the RCE receiver) to activate the vigilance control valve on the locomotive to simulate a vigilance penalty, resulting in de-excitation of the main generator and an emergency brake application. The mechanism was not designed under recognised system safety principles (see Appendix B). It was removed by TasRail with the introduction of generation 2 RCE. TasRail advised that it relied on the RCE emergency stop function as having sufficient safety integrity (see Safety integrity).
In addition, ADE advised that all 3 generations of RCE were compatible with the fitment of a stop button on the locomotive, external to the RCE. If fitted and then activated, this device would cause the RCE to vent the brake pipe and remove traction power. The external stop button was not used.
Failure mode reset procedures
The RCE could be reset in 3 ways:
resetting the transmitter and receiver from communication failed mode by configuring the transmitter (with direction in neutral, throttle idle, independent brake fully applied, and automatic brake set) and then pressing the train set button on the transmitter (Figure 13)
resetting the receiver from control fault mode or failed train brake mode by cycling the transmitter power
restarting the receiver completely by either cycling the receiver’s isolation circuit breaker or removing and reinserting the TR class locomotive’s multiple unit electrical cable into the RCE receiver.
ADE advised the ATSB that the RCE was designed to facilitate a fast communication failed mode recovery when the train was travelling at speed to avoid stopping the train.
Inspection and maintenance
The generation 3 RCE was serviced every 30 days by an external contractor at Devonport. To ensure continuity of service, TasRail had purchased 2 complete RCE units, which comprised of a receiver box, an air box and a transmitter. When the in-service unit was removed for servicing, it was replaced by the spare.
There were also requirements (from both TasRail and ADE) for the units to undergo regular in-service testing. Although some of ADE’s requirements were not specified in the RCE operation manual, it had advised TasRail of these test requirements in correspondence 3 weeks after generation 3 RCE commenced operation (February 2018). The RCE in-service tests specified by ADE are compared with those that TasRail required in Table 2.
Table 2: Generation 3 RCE in-service test intervals
RCE tests
Test intervals
ADE advice
TasRail instruction
Vigilance function
Shift change[1]
First entering service and after locomotive shutdown[2]
Tilt function
Shift change[1]
Not specified
Emergency stop
30 days[1]
Not specified
Railton loading interlock
Not specified
30 days
General functionality:
independent brake
automatic brake
emergency stop
direction test, with power
After a cold start of the receiver (following power removal)[3]
Not specified
ADE advised TasRail of these test requirements in correspondence 3 weeks after generation 3 RCE had commenced operation.
A locomotive shutdown involved the engine being turned off and the battery switch opened (for example, during track closures).
Specified in the RCE operation manual.
Neither the RCE operation manual or TasRail required post-failure troubleshooting or testing to occur after a communication failed mode, control fault mode or failed train brake reset. Therefore, there was no requirement to ensure correct functionality and safety of the RCE prior to continued use after a fault mode had occurred. TasRail drivers also advised that, after experiencing an RCE fault, functionality tests were not routinely conducted.
TasRail advised the ATSB that testing of the interlock at Railton was not routinely performed. Additionally, several drivers advised that the tilt function test was performed on initial RCE setup or after a locomotive shutdown rather than at shift change. The driver of train no. 604 advised that they performed the general functionality test every 7 days or after a locomotive shutdown. There were no records provided to the ATSB documenting when tests were conducted, and it is believed their performance were not routinely recorded.
Remote control equipment safety considerations
Overview
As previously discussed and detailed in Appendix A, analysis of available evidence identified that, during the accident sequence, the RCE became unresponsive to driver commands at the point of a fast direction change from forward to reverse. The RCE tried to oppose a locomotive-induced brake application after the RCE fault state cleared, and then triggered 2 emergency brake applications and releases, which did not replicate any documented fault condition behaviour.
Hardware tests after the accident indicated that there were no permanent hardware faults associated with the RCE.
Accordingly, the ATSB analysed the design, behaviour and integration of the RCE in detail using test results, design documentation, software code, interviews with ADE, and other information. During these activities, design and integration issues between the RCE and cement train consist were identified. Most of these related to the way ADE had designed the generation 3 RCE to interact with and control the operation of the braking systems outlined in Train braking systems.
This section discusses a range of RCE design and integration problems that ATSB testing and analysis identified.
Reduced effectiveness of some safety features
The RCE replicated several safety features that were available to a driver operating a non-remotely-controlled train. However, some of the safety features were absent or had reduced effectiveness when a driver was using RCE to control the train, and some were absent when the driver was not in the locomotive cab (that is, using the RCE from outside the train or within the driver’s van). A list of safety features that were available to the driver of a non-remotely-controlled train (when operated from the TR class locomotive cab) is provided in Table 3. A comparison of whether these functions were also available under RCE control, when the driver was operating the train from specific locations, is also provided.
In addition, the efficacy of several of the safety features that were replicated during RCE operations were altered. TasRail could not provide evidence that risk assessments were performed for the absence, suppression, alteration or limited access to the safety features listed in Table 3, as a result of RCE operations (see also Risk assessments for remotely-controlled train operations).
Table 3: Summarised driver controls and selected safety features
Reduced effectiveness of automatic emergency brake
As explained in Emergency vent valves, the TR class locomotive’s VX vent valves and emergency magnet valve (EMV) were designed to locally exhaust the brake pipe on sensing a rapid rate (emergency) brake pipe pressure reduction.[42] In addition, the locomotive’s braking system would cease recharging the brake pipe. These actions assisted propagation of both driver-commanded and uncommanded emergency brake applications. Further, once the brake pipe pressure was less than about 297 kPa, the locomotive would activate the power control switch (PCS), removing all traction power.
The ATSB found that the RCE receiver would not recognise brake applications (including emergency applications) it did not command. Instead, it would oppose uncommanded reductions in brake pipe pressure (for example, as the result of a derailment or train separation) and attempt to restore the brake pipe pressure to 500 kPa. As a result, following an emergency brake application at the driver’s van, the locomotive’s emergency brake response would not activate. Tests conducted by the ATSB by making an emergency brake application from the driver’s van without the RCE fitted also showed that, even without the RCE opposing the brake pipe pressure drop, there was insufficient pressure drop at the locomotive to activate the locomotive’s emergency brake response.[43] Tests of emergency brake application at the driver’s van with the RCE active resulted in the following effects:
the driver’s van almost immediately enacted an emergency brake activation as a maximum service brake application at the front of the train
it took about 40 seconds for a minimum-service application to take effect at the locomotive
the brake pipe pressure at the locomotive remained above 425 kPa after 60 seconds
the locomotive’s VX vent valves and EMV did not activate
the locomotive did not invoke the locomotive’s power control switch (PCS).
Had there been an event where the driver activated the driver’s van emergency brake (for example, during a fail-to-unsafe event of the RCE) with the locomotive propelling under traction power, it would have been largely ineffective. As such, the 108-t locomotive would have remained under traction power pushing into 18-t empty wagons with high braking force at the driver’s van and reduced wagon braking force closer to the locomotive. As explained in Propelling operations, the resultant buff forces originating from the rear of the train consist would have caused increased lateral force against the rail head and a reduction in wheel load, increasing the risk of a mid-train jack-knife derailment.
ATSB tests showed that either of the following 2 actions resulted in uncommanded emergency brake applications that would clear without driver intervention:
rapid RCE automatic brake commands past the minimum service position (when the train was configured as a light engine)[44]
rapid movement of the RCE direction controller to the opposite direction.
The rapid decrease in brake pipe pressure resulted in an emergency brake application due to activation of the locomotive’s VX vent valves and, in turn, EMV. In addition to assisting the brake pipe pressure drop, the locomotive’s airbrake system would activate the locomotive’s power control switch (PCS), removing the locomotive’s traction power.
In this scenario, the RCE would not recognise the brake pipe venting and continue to attempt to recharge the brake pipe, as explained in Reduced effectiveness of automatic emergency brake. This would then lead to the brakes being released after the initial application.
A rapid brake command past the minimum service position was only observed to result in an emergency brake application when the train was configured as a light engine. This was very likely due to the greater volume of brake pipe air from a train consist leading to a slower reduction of brake pipe pressure during venting, preventing a trigger of the locomotive’s VX vent valves.
However, a fast direction change could result in an emergency brake application in any configuration (that is, with the locomotive leading the consist, trailing the consist, or configured as a light engine). The fast direction change resulted in a momentary trigger of the dual direction fault interlock, which rapidly exhausted brake pipe pressure.
When the test was conducted with the RCE integrated into the TR class locomotive, the momentary exhaust created a pressure reduction rate in the brake pipe sufficient to trigger the locomotive’s VX vent valves. This subsequently activated the EMV on the locomotive and both continued to vent the brake pipe after the initial RCE fault had cleared.
When the ATSB tested the RCE in isolation on a passive test bench, the momentary exhaust of brake pipe air through the RCE emergency brake valve caused little pressure loss and the desired brake pipe pressure was quickly restored (that is, the response on the test bench was different to that on a train). Similarly, ADE reported that the condition could not be replicated during later bench testing.
ADE mistakenly believed VX valve activation was caused through an electrical command by software (in fact it was through rapid automatic brake pipe pressure reduction). ADE reported that it did not have any information in relation to the VX vent valves.
The issue of uncommanded emergency brake applications on light engines was highlighted to TasRail in correspondence by ADE in April 2018, and this integration concern was communicated in an email to RCE drivers in the following month.
Wheel locking in communication failed mode and later reduction of braking effectiveness
When using generation 2 RCE, if the radio link was lost when the train was moving under traction power with brakes released, the RCE receiver would (among other things) apply 325 kPa control pipe pressure and reduce the brake pipe pressure to 350 kPa (full service). Although this worked well with the DQ class locomotives, the communication failed mode was causing skidding and flat wheels on the then-new TR class locomotives. The TR locomotives had been introduced at the same time as new wagons (from HE class to THFY class), and it was suspected that the skidding problem was associated with the resulting changes in the braking system from the previous locomotive and wagon types.
ADE diagnosed the cause of the skidding problem to be the rapid exhaust of brake pipe pressure by the air box, which was activating the TR class locomotive’s EMV, resulting in an emergency brake application.
In response, ADE changed the communication failed mode response in generation 3 RCE. Under the new response, the RCE would now reduce the brake pipe pressure to 400 kPa (less than a full-service application) and only set control pipe pressure to 325 kPa if it was already less than 375 kPa when the communication failure occurred (otherwise it would maintain the current control pipe pressure). There were no reports of locomotive flat wheel events related to communication failed mode events after the changes were implemented.
The TR class locomotive, unlike the DQ class locomotive, had an electronic brake control system. A feature of the system was the ability to apply increased locomotive brake cylinder pressure in response to control pipe pressure, known as the ‘J-multiplier’ effect. When the control pipe was charged to the maximum of 350 kPa, the locomotive brake cylinders would be charged to about 550 kPa.
Under the generation 2 RCE penalty mode, when the receiver entered the communication failed mode and commanded a maximum of 325 kPa in the control pipe, the TR class locomotive electronic brake control system responded by applying 520 kPa to the locomotive’s brake cylinders. This brake cylinder pressure (520 kPa) was about 50% higher than what would be achieved during an automatic brake full-service application (that is, 350 kPa). The full-service brake cylinder pressure was designed to be the highest level achievable on the TR class without risk of wheel lock in normal conditions.
Therefore, the ATSB found that the likely reason for the reported locked wheels was that the generation 2 RCE communication failed mode applied a significantly higher pressure in the locomotive brake cylinders than the maximum brake cylinder pressure that could otherwise be made (520 kPa vs 350 kPa).[45]
Further, testing conducted by the ATSB showed that an unintended emergency brake application could occur on a light engine, but not when the locomotive was attached to a train as was being reported. This explains why ADE’s change to control pipe behaviour during the communication failed mode response prevented the generation of unintentionally high locomotive brake cylinder pressures and corrected the issue of flat wheels.
Uncommanded release of light engine automatic brake applications
As discussed previously, when configured as a light engine, rapid RCE transmitter automatic brake commands past the minimum service position resulted in unintended activation of the locomotive’s VX vent valves and (in turn) EMV. This resulted in an uncommanded emergency brake application. The RCE’s attempts to attain the selected brake pipe pressure would result in a partial release of brakes on a light engine. A full release was avoided due to the RCE seeking a pressure of 65 kPa in the control pipe during an automatic brake application.
The ATSB observed further issues during testing of the communication failed mode. In circumstances of a released independent brake at the initiation of a communication failed mode event, as may occur during light engine shunting operations, no control pipe pressure was commanded by the RCE. As a result, the restoration of the brake pipe pressure to 400 kPa after the uncommanded emergency brake application, which happened automatically, resulted in the release of the locomotive’s brakes as the brake pipe pressure rise was a valid automatic brake release command. That is, although the transmitter displayed a brake pipe pressure of 400 kPa to the driver (indicating a brake application was in effect), the locomotive’s brakes would be fully released.
Uncommanded release of wagon brakes
Following a locomotive engine restart after an automatic engine shutdown,[46] there would be a momentary loss of electrical power that would result in a cold start of the RCE (in the same way as cycling the power). The RCE receiver would power down, resulting in the brake pipe being vented and the brakes being applied across the train. When electrical power was restored, the RCE would restore the brake pipe from 0 kPa to that commanded by the driver on initialisation (for example, 350 kPa or full service). The brake pipe pressure rise was similar to a valid automatic brake release command and therefore resulted in a full brake release on all wagons.
In this situation, the RCE transmitter display screen would show restoration of brake pipe pressure to the driver’s originally commanded automatic brake application (that is, less than 500 kPa), indicating to the driver that the wagon brakes were still applied.
As the independent brake was fully applied during a cold start / engine restart after automatic engine shutdown, a train runaway was normally prevented.
Uncommanded release of locomotive independent brake
Dynamic brake control was an addition to the generation 3 RCE that had not been present in previous generations. Shortly after introduction, drivers found that engaging dynamic brake on the transmitter would result in an uncommanded release of the locomotive’s independent brake. This was a design feature of the TR class locomotive that, ADE advised TasRail at the time, was replicated by the RCE.
The feature to which ADE was referring was the ‘dynamic brake interlock’ function. On application of dynamic brake, this function would release service rate brake cylinder pressure on the locomotive that was applied by the automatic brake (in order to prevent wheel lock). The locomotive dynamic brake interlock would not release brake cylinder pressure applied by the independent brake, while the RCE did release with independent brake application.
The effect of this integration issue was the potential for a train, being held stationary by the independent brake on a gradient, commencing to roll away uncommanded when dynamic brake mode was selected (for example, after completion of loading at Railton).
RCE dynamic brake was suspended from use by TasRail at the time of the 21 September 2018 runaway due to an unrelated issue (see Issues after commissioning).
Remote control equipment emergency brake reset
As previously discussed, the air box was relocated to the rear of the locomotive with the introduction of the TR class during generation 2 RCE (see Figure 12). This resulted in a greatly increased recharge rate of the brake pipe at the locomotive (smaller pipe air volume forward of the air box) compared to the rest of the train’s brake pipe (larger pipe air volume behind the air box).
Testing conducted by the ATSB found that, on a non-remotely-controlled train of 220-m length, the locomotive registered a brake pipe rise from 0 kPa to 495 kPa in 49 seconds. On the 220-m long cement train with the RCE supplying air from the back of the locomotive, the same pressure rise for the locomotive took only 16 seconds. Although the recharge rate at the locomotive increased, the recharge of the brake pipe on the trailing portion of the train was not proportionally increased.
In effect, this meant that traction power, which was deactivated once the brake pipe pressure had risen sufficiently, could be regained much sooner on a remotely-controlled train than on a non-remotely-controlled train. Once this had occurred, the driver could reapply traction power.
In addition, the TR class locomotive’s braking system prevented a reset of both driver-commanded and uncommanded emergency brake applications for 60 seconds. This ensured the train came to a complete stop before a brake release was attempted, which prevented excessive draft forces and possible train separation, as explained in TasRail’s train handling manual. Conversely, the RCE allowed a reset of the transmitter’s commanded emergency brake application in less than 30 seconds by cycling transmitter power.
In combination, the faster reset of the traction power inhibition once the brake pipe began recharging, and the faster pressure rise at the locomotive, resulted in a significantly shorter time between when the emergency brake application occurred and when power could be reapplied. The reduction was from about 1 minute 49 seconds on a non-remotely-controlled train to 46 seconds on the cement train fitted with generation 3 RCE.
Allowing the reapplication of traction power in a shorter time reduced the protection against excessive draft forces. The reduced time was likely to have been faster than the time required to release the fully-applied automatic brake along the remaining length of the train, raising the possibility of having the locomotive pull against applied wagon brakes, further increasing draft forces.
Software design
There was no documentation associated with the RCE’s software, apart from limited comments in the software code (which sometimes contradicted the code).
An ATSB review of the code indicated that a state could exist during initialisation where the receiver would not do anything other than maintain previous commands. Specifically, there were several initialisation tests which, depending on certain conditions or in the event of a single hardware or memory fault, had the potential to enter an endless loop (that is, the receiver could ‘hang’) without entering a fault state. Among the conditions that would result in this behaviour during initialisation were when the RCE receiver detected a non-neutral direction, measured train brake pipe pressure above 470 kPa,[47] or measured independent brake control pipe pressure below 300 kPa.
In this state, the receiver would not issue new commands in response to driver inputs, and would not enter fault states such as the communication failed mode. Recovery from this state was only possible through external correction of the conditions that caused it (such as the train brake pipe pressure dropping below 470 kPa via an external means). Even if power was cycled, the RCE would re-enter this state if the external condition remained unchanged.
ATSB review of the software indicated that the functions to set the train and RCE in a desired initialisation state (such as a set brake pipe pressure or direction selection) were not called before this loop, although this could not be tested.
Other issues identified during ATSB testing
In addition to several integration and design problems identified between the RCE and a train consist, the ATSB also noted the following instances of uncommanded (that is, not the result of driver input) locomotive behaviour during testing (see Appendix A) and subsequent review of the recorded data:
The locomotive registered multiple instances of uncommanded dynamic brake activation, or light traction motor current, sometimes in conjunction with an electrical control fault mode occurrence.
During one instance of an electrical control fault mode occurrence after conclusion of an uncommanded dynamic brake activation, the engine speed increased and the generator field activated in conjunction with light traction motor power. However, the direction controller had remained in the neutral position during this time. That is, these commands were not the result of driver inputs.
The ATSB could not determine the origin of these uncommanded dynamic brake and light traction motor currents. However, the ATSB noted:
the uncommanded dynamic brake activations were not observed during periods when the RCE was not active
the uncommanded light traction motor currents were still observed shortly after the RCE was made inactive.
Caroline Creek event
The ATSB also reviewed driver reports of faults (see Fault reporting process) and related information to identify any potentially similar events involving the generation 3 RCE to that which occurred at Railton on 21 September 2018. There were several events which indicated anomalous RCE behaviour. The most significant of these occurred at Caroline Creek in May 2018.
At 2214 on 8 May 2018, a network control officer (NCO) received an emergency alarm from cement train no. 311 while en route from Devonport to Railton. The driver advised the NCO that they had received an RCE emergency penalty and were attempting to reset it.
When several attempts to reset the transmitter had failed to restore control, the driver determined that the receiver would need a cold start to reset the fault. However, the cement train was straddling the Caroline Creek bridge, which had no walkway fitted. This meant that the driver, who was positioned in the driver’s van at the front of the train, required someone to attend the locomotive at the rear of the train to perform a cold start of the RCE receiver.
At 2237, the driver advised the NCO:
…no good here I’m afraid, I can’t get this thing to build up brake [pipe pressure] or anything. It builds up about 300 kPa of brake pipe pressure then it’ll drop out, then I try and reset it and it goes to control faults [control fault mode], then it’ll come back up again [brake pipe pressure] then it goes to control failures [control fault mode].
At about 2242, while assistance for the driver was being arranged, one attempt at a reset of the fault was successful, for reasons that were unclear. The cement train continued the journey to Railton shortly afterward.
As the event resulted in significant delay to the cement service, TasRail contacted ADE for assistance in identifying the fault condition. ADE replied that for a suspected control fault mode to reset, the following was required:
clearance of the fault condition causing the control fault mode
brake pipe pressure at or below 350 kPa (full service or greater)
TasRail subsequently provided ADE with a screenshot of the locomotive’s recorded data from the event and a copy of the verbal conversation between the driver and NCO to assist in diagnosis of the cause that had resulted in the suspected control fault mode event. TasRail advised the ATSB that ADE later gave a sufficient verbal explanation for the event, which led to continued use of the RCE. No records were provided to detail what this explanation was and ADE later advised the ATSB that the cause of the vent could not be determined with certainty.
Analysis of the event at Caroline Creek event was conducted by the ATSB. The analysis utilised data recorded on the locomotive during the event and the information recorded during the driver’s interaction with train control on the day. The ATSB identified that during the event:
The train experienced an uncommanded emergency brake application followed by rapid and repeated brake pipe and brake cylinder pressure fluctuations. These observations were not consistent with any fault condition behaviour documented for the RCE.
The generation 3 RCE likely released the independent brake and attempted to recharge the brake pipe on multiple occasions.
The brakes remained applied on the locomotive due to systems external to the RCE; specifically, the locomotive’s electronic airbrake system held locomotive brake cylinder pressure until brake pipe pressure rose above 350 kPa.
Had the RCE successfully continued to increase the brake pipe pressure above 350 kPa in the absence of control pipe pressure, the locomotive brakes would have released without command, leaving the train in an un-braked state.
The unknown fault condition was cleared by an uncommanded automatic locomotive engine restart, which likely cold-started the RCE receiver. Upon resumption from the cold start, the receiver did not resume the unknown fault condition and subsequently began accepting control messages from the transmitter.
Both the behaviour of the generation 3 RCE while a fault state was present (attempting to remove independent brake applications and recharge the brake pipe), and that external systems were required both to maintain locomotive brakes and cold start the RCE receiver, are indications that the RCE had entered a potentially unsafe state.
In addition to the event that occurred at Caroline Creek on 8 May, the ATSB conducted analysis of recorded data from other selected generation 3 RCE fault reports from drivers. Reports were selected on the basis of driver descriptions of RCE behaviour that could not be readily attributed to a defined failure mode or response. These are summarised below:
16 February 2018: The driver reported mismatches between RCE transmitter display and locomotive display (likely transmitter screen freezes), commanded brake releases not effected by the RCE, slow response by the RCE to commands, and multiple communication failed mode events during their shift.
ATSB analysis: Recorded data indicated that at least one of the reported communication failed mode events was likely a control fault mode event, based on the observed brake pipe pressure response. There was insufficient information available to identify the reports of mismatch between RCE and locomotive displays and between driver command and system response (brake releases not effected and slow response).
21 February 2018: The driver initially reported a non-descript RCE fault, and later advised that continued use of RCE was ‘unsafe’ after inability to reverse the consist.
ATSB analysis: Recorded data showed direction returning to ‘centre’ (loss of direction selection while train was in motion with throttle applied). After the train had stopped, multiple instances of high throttle notch without direction selection occurred, corresponding to the time of the initial report. As the direction was in neutral, no traction power was applied. Further instances of throttle applied without direction selection occurred prior to the second report. No reason for the initial loss of direction, or subsequent non-applied direction, could be observed from the recorded data as the direction commanded by the driver on the RCE transmitter was unknown.
13 April 2018: The driver reported that the train had come to an uncommanded stop requiring a full locomotive and RCE restart to recover, and subsequently slow RCE response to driver commands while loading.
ATSB analysis: Recorded data showed that the uncommanded stop was likely due to a RCE communication failed mode event (based on brake pipe and brake cylinder pressure behaviour). The slow response to RCE commands could not be directly observed as commands were not recorded. However, multiple instances of maximum throttle settings and high traction motor amperage, with no associated train movement, were observed during loading and therefore likely corresponded. There appeared to be attempts by the driver to reset dragging wagon brakes through the application and release of full-service automatic brakes. It is unclear whether this was related to or resolved the issue the driver reported.
19 May 2018: The driver reported an uncommanded emergency brake application with simultaneous uncommanded horn operation.
ATSB analysis: Multiple instances of emergency brake pipe pressure decreases were observed. The reason for these system behaviours could not be determined from the available data.
3 June 2018: Multiple reports. An emergency Selcall received by the network control officer, and the driver reported the train had stopped while en route with a fault (initially reported as a control fault mode event then as multiple communication failed mode events).
ATSB analysis: Although the RCE was capable of requesting the locomotive to send emergency Selcalls, the activation parameter was not recorded. There were no brake pipe reductions recorded consistent with an emergency or vigilance penalty corresponding to the time the Selcall was received. Multiple events, consistent with the RCE communication failed mode (based on brake pipe and brake cylinder pressure behaviour) occurred while the train was en route over a half-hour period, corresponding to the time of the driver’s report.
In addition, a review of the RCE driver fault reports found 25 references to uncommanded emergency brake applications or alarms being reported. In at least 8 of these, the train incurred an RCE-induced emergency penalty without driver command. The reasons for these were not determined, and were not found to be the result of any emergency penalty condition described in the RCE operation manual or the failed train brake application fault mode.
The data recorded for these events was limited (see Effect of non-recording of remote control equipment data), so confirmation of reported faults was generally not possible.
Summary
In summary, the ATSB testing and analysis identified several design and integration problems associated with the generation 3 RCE, including problems that could result in unintended brake application and unintended brake release.[49]
Apart from the uncommanded independent brake release during the use of dynamic brake and reduced communication failed mode braking, neither TasRail or ADE had identified any of the other RCE integration and design problems. Further, for these 2 problems that had been identified, Tasrail could not provide any evidence that a risk assessment was performed to assess the impact and immediate effect they may have had on the safety of its RCE operations (see also Risk assessments for remotely-controlled train operations).
Remote control equipment development
Overview
Given the problems identified with the RCE’s design and integration, the ATSB investigation examined aspects of the RCE’s design and development process.
The Rail Safety National Law (RSNL) (see Relevant legislation) stated that rail safety was a shared responsibility between various parties, including designers / manufacturers and rolling stock operators (as well as other parties such as rail safety workers and the regulator).[50] The RCE manufacturer was Air Digital Engineering (ADE). TasRail was the rolling stock operator that commissioned and then used the RCE.
Rail safety duties applicable to designers, manufacturers and suppliers of rolling stock were contained in section 53 of the RSNL. It stated in part that, so far as was reasonably practicable, rolling stock:
was safe for its intended use
was tested and examined to ensure safety for its intended use
had information available for its correct use, including any conditions
had results of testing and examination available.
Rail safety duties applicable to rolling stock operators were contained in section 52(4) of the RSNL. It stated that a rolling stock operator must ensure, so far as is reasonably practicable, a number of things including:
the provision or maintenance of rolling stock that is safe; and
that any design, construction, commissioning, use, modification, maintenance, repair or decommissioning of the operator's rolling stock is done or carried out in a way that ensures safety …
This section of the ATSB report provides a background on ADE. It also details aspects of the design process for the RCE and the relevant ADE and TasRail interactions and activities throughout the development of the generation 3 RCE.
Information about the service history of the generation 3 RCE is provided in Issues after commissioning and Remote control equipment fault reporting. Information about TasRail’s management and oversight of the generation 3 development project is provided in TasRail change management and related processes. Further information about regulatory background and oversight is provided in Regulatory oversight.
Manufacturer information
ADE is a small Australian-based company specialising in the design and manufacture of RCE used to control railway locomotives. Throughout development of the 3 generations of RCE, the company primarily consisted of one person, who owned the company, and conducted most of the hardware design, manufacture, and customer liaison work. Other individuals were utilised on an as-needs basis, such as a programmer who developed software code for the generation 3 RCE, and an electronics company in Devonport that was subcontracted to provide local maintenance support for TasRail on behalf of ADE.
External guidance
In 2005–06, the United States Federal Railroad Administration (FRA) published a series of reports addressing the risk of remotely-controlled locomotive operations in general (FRA 2005, 2006a, 2006b, 2006c). At the time there were ‘limited’ remotely-controlled main-line train operations (FRA 2005), so the reports primarily addressed yard operations. Collectively, these reports highlighted a range of different risks, notably situation awareness, operator training and procedures, equipment reliability, and radio security.
There was a broadly similar accident rate in yard operations between remotely-controlled trains and conventional (non-remotely-controlled) trains in the United States (FRA 2005). Injury rates tended to be lower in remote operations, which was proposed to be the result of a reduction in crew size.
FRA (2005) discussed system response to a loss of radio communications, stating concern about the amount of control afforded to the driver in such situations:
For example, there have been incidents in yards where the [remote controlled train] suddenly stopped because of communication failure and caused a section of the cars being handled to break away. In one instance these cars rolled into the side of a train, causing a derailment. To have such occurrences on high-speed main tracks could prove catastrophic. FRA recognizes that penalty brake applications can and do occur to engineers [drivers] during conventional main track operations. However, the engineers have the ability to immediately respond to these situations with considerably more controls than those afforded to [remote controlled train operators].
Likely in part due to the limitations of the available equipment at the time in the United States, the FRA recommended limits on locomotive power, train length, speed, and grades for remotely-controlled trains, effectively recommending their use be limited to yards and short distances.
There were no design standards or guidance documents specifically developed for the design of RCE for trains. However, there are standards and guidance documents to aid in the system safety approach for rail systems (see Appendix B).
Safety integrity
Software integrity level and Australian Standard 61508
Functional safety is a concept relating to the need for a system to function correctly to maintain safety. Australian Standard (AS) 61508 (Functional safety of electrical/electronic/programmable electronic safety-related systems) is a functional safety standard. It provided suppliers of components and subsystems with complete life-cycle processes to optimise system safety through risk-based and performance-based measures (see Appendix B).
Safety integrity was defined by AS 61508 as ‘the probability of a safety-related system satisfactorily performing the required safety functions under all the stated conditions within a stated period of time.’ The concept of ‘safety integrity level’ (SIL) was used by a number of standards, including AS 61508 and EN 50126.[51] Generally, SIL was a number specifying the safety integrity requirements of safety functions of a system. AS 61508 defined SIL ranges from 1 (the lowest defined level of safety confidence) to 4 (the highest).[52]
In correspondence with TasRail in February 2018, ADE stated:
The [RCE] is designed with failsafe features following AS61508.
ADE reported that the RCE had always been designed with the objective of being fail-safe; that is, so that any hardware or software failures would always result in a safe state. TasRail advised the ATSB that, in relation to this aspect and the design of the RCE more broadly, it (as a consumer) necessarily placed significant reliance on the supplier’s skill, expertise and representations.
The pre-delivery and commissioning testing report for the generation 3 RCE, dated 8 January 2018, defined ‘safe state’ as:
Disabled propelling [traction] and brakes applied.
ADE later advised the ATSB:
The emergency stop function on the Transmitter has always been the critical safety function that must work when required provided it is used at the correct time. In support of this claim, attached is the SIL testing documentation (Estopsil) undertaken in 1999 when the systems were designed and tested in accordance to AS61508. The philosophy of the design then has not changed relating to the use of Emergency Stop function.
The document referenced by ADE was Estopsil (an abbreviation of Emergency Stop SIL), authored by ADE and dated April 1999. The document described the assessment of safety of the emergency stop function of an RCE system developed in 1997, which was presumably the generation 1 RCE provided to TasRail in 1999. It was the only formal document produced that was relevant to the claim of design and test in accordance with AS 61508. The 31-page document contained information relating to a ‘safety integrity level’ (SIL) of the emergency stop safety function.
ADE also referred to a SIL in the generation 3 RCE operation manual:
The safety functions that exist and operate within the equipment are provided for the maximum safety integrity level of the remote control equipment.
Although it is almost certain that the generation 3 RCE software was tested to some extent, there were no records of what was tested and to what extent. Records that were provided by ADE related to the generation 1 RCE.
Safety integrity scope
ADE’s application of AS 61508 was primarily applied to the emergency stop function (described in Emergency stop function), with limited information about the RCE’s other functions and processes.
The February 2018 correspondence from ADE to TasRail stated:
The primary safety mechanism relating to the AS61508 Safety Integrity Level is the Emergency Stop function.
The Emergency Stop operation meets SIL 2 and SIL 3 provided it is tested periodically not more than 30 days for a high usage demand mode which the equipment would come under.
The Estopsil document contained:
assessments of the dependability of the emergency stop function
a flowchart of the software development process (which did not include any risk or safety analyses)
a radio communications state diagram
a table providing ‘alternative pathway[s] that would be used’ in the event of failure (for radio failure, the alternative was ‘emergency stop or redundant radio’).
The assessment did not consider external factors that could prevent the emergency stop function from working (such as radio communication, software, or hardware failures).
Assumption of acceptable dangerous failure probability
The Estopsil document stated an assumed acceptable dangerous failure probability of 1% per year of operation. There were no records indicating whether TasRail provided this as a specification to ADE, was informed of ADE’s assumption, or later accepted this nominal safety acceptance criterion and assimilated it into its broader safety management system.
Software assessment
In relation to software modules associated with the generation 1 RCE, the Estopsil document stated:
To test every [software] module individually for the preparation of this particular document is an impractical task for the following reasons:
Necessity to provide documentation at some point in time (preferably sooner) of testing the safety functions required to operate on demand in dangerous conditions rather than safety functions that continuously operate in high visible form.
To provide accurate calculation and the validation of the probability of a dangerous failure per operational hour.
In this case, it is believed more advantageous to collectively test a system with mainstream module structure and commonality of hardware.
The selection of the Emergency Stop to be tested was based on the fact that this single function is designed and displayed to the equipment user as the most effective means to generate a safe state from a potential dangerous state. This safety function is not uniformly cyclic as is others within the equipment and in judgment of these criteria this function becomes the subject of a targeted failure measure in both high & low demand mode of equipment operation.
Overall safety integrity assessment
The Estopsil document applied some concepts and methods from AS 61508 to the generation 1 RCE. The document stated:
There is validity to the question that although any equipment has been tested in correct operational state, how reliable would any safety function be given that under fault conditions of predetermined foreseeable failures of system components, the safety mechanisms designed into the E/E/PE[53] equipment fail to protect as intended or render the E/E/PE in a dangerous state.
In realistic term, to address these matters, reference has been drawn from the Australian Standard 61508 and various parts that allow a determining factor to be assessed of what is now accepted and termed a Safety integrity Level (SIL)…
The documentation provided herewithin contains only relative information to performance of the safety function being tested and does not include explicit detail such as part numbers, circuitry structure, detailed software code or similar. Most of these items are a variable format on a system by system basis, however the pathways of execution remain somewhat consistent.
The document indicated that tests had been performed in support of reaching the SIL determination, recorded to have taken place on 1 December 1997 and 1–11 March 1999. The tests included the following:
analysis of the effects of various faults in hardware logic related to the emergency stop function
a test involving ‘catastrophic’ (complete) failure of the radio module, including power loss, which would result in an emergency stop activation. This was later contradicted by a communications flow chart that did not include this functionality.
a test that concluded that ‘a failure of the software component results in [activation of the hardware-induced emergency stop function] due to the watchdog not being acknowledged’
an assessment of hardware reliability, which provided incomplete data related to failure rates, failure modes, and diagnostic coverage.
The document used some elements of the standard to perform basic analysis of the emergency stop function. It contained no information or evidence about the application of AS 61508 throughout the system’s life cycle. For example, there was no assessment of the potential consequence of equipment failure or an evaluation of development practices against the recommended and highly recommended practices for each SIL defined in the standard.
The Estopsil document included notes about prospective design changes, such as the following:
Initial test’g showed need to split double h/w opts to spread dependency from single h/w IC to two.[54] This can be done at the system design phase.
There was no explanation for why this change was necessary (for example, whether it exceeded some nominal failure rate maximum). There was also no information regarding whether the change was implemented, or whether doing so addressed the underlying problem.
Other than the 1999 Estopsil document there were no other records that would necessarily result from a design process complying with AS 61508, such as requirements specifications, hazard and risk analyses, and a safety case. This was the case for the generation 2 and generation 3 RCE as well as the generation 1 RCE.
In summary, ADE used elements of a system safety standard, AS 61508, to estimate the reliability of the emergency stop function for the generation 1 RCE in 1999 but this was not equivalent to having developed the equipment to that standard. The generation 3 RCE incorporated new functions, software, and radio and microprocessor hardware (see Differences from previous generation). Although such changes would necessitate re-analysis, there was no documented evidence that this was conducted.
Operations with previous remote control equipment
As previously noted, ATN Tasrail trialled ADE’s generation 1 RCE in 1999. ATN Tasrail’s planning manager (who was a qualified train driver) operated and tested the generation 1 RCE on the cement train for 4 months. Following these trials, ATN Tasrail commenced using the RCE. All Devonport cement train drivers were trained in its use, with a designated pool of drivers later formed under the supervision of the planning manager. TasRail personnel advised the ATSB that RCE operations initially operated in conjunction with a 2-person crew configuration to allow familiarity, before changing to a driver only operation configuration. The planning manager retained supervision of the RCE driver team until a restructure in about 2009.
Generation 1 RCE operated without serious incident from its commencement in late 1999 until February 2010, when the RCE failed to an unsafe state. At the conclusion of a light engine movement, the driver brought the engine to a stop by applying full independent brake, removing traction power, and placing the direction controller in neutral. Shortly afterwards, the DQ class locomotive commenced an uncommanded powered movement while sounding the horn and dropping sand. It travelled in reverse for 150 m before colliding with a rake of stationary cement wagons at 15 km/h and continued to push into the stationary wagons after the collision. The driver took evasive action, including applying the emergency brake and switching off the transmitter but these actions had no effect. To stop the locomotive from continuing to power into the wagons, the driver removed the multiple unit plug from the receiver (on the locomotive) to cease the traction power command.
Subsequent investigation by ADE attributed the accident to an electrical short from a stray piece of metal within the receiver, causing all outputs through the multiple unit plug to be erroneously active. Issues with stray voltage to ground through the RCE antenna due to damaged cable insulation were also found. As a result, ADE introduced the dual direction fault interlock function into the generation 2 RCE.
As previously noted, TasRail replaced the generation 1 RCE with generation 2 RCE in about August 2010. After this period and until the generation 3 RCE project, TasRail did not have a nominated responsible person or team for the management of the RCE operation. TasRail advised that a shift from individual responsibility to departmental responsibility was consistent with modern practice.
Generation 3 remote control equipment project initiation
In early 2015, TasRail identified issues with decreasing reliability of generation 2 RCE. It initiated discussion with ADE regarding an increased maintenance regime to extend the RCE’s operational life. However, ADE advised TasRail that generation 2 RCE could no longer be supported going forward due to discontinued manufacture of the transmitter and receiver microprocessors.
As a result, TasRail staff researched RCE replacement options. In consideration of these options, it was decided to approach ADE for a replacement, described by one involved TasRail employee as ‘…a like for like replacement of the existing [RCE]’, including additional options.
In late May 2015, ADE provided TasRail a quote for one new RCE unit sourced externally (from a United States-based manufacturer), as ADE was reluctant to manufacture its own product due to the level of work involved. The externally-sourced RCE was substantially different to the ADE-manufactured generation 2 RCE.
In November 2015, funding became available for TasRail to purchase the new RCE, at which time a further quote was requested from ADE, with the addition of dynamic brake control via the RCE (see Dynamic brake) added to the required functionality. ADE provided the requested quote, again with RCE sourced externally, with a 12-week turnaround period. A GPS speed restriction option was also offered at additional cost.[55]
In late 2015, the Australian Communications and Media Authority (ACMA) advised TasRail of the need to vacate the generation 2 RCE radio spectrum allocation by April 2017, due to its reallocation to emergency services. ADE advised TasRail that the generation 2 RCE could not be reprogrammed with the newly-allocated radio spectrum. As a result, in December 2015, TasRail advised ADE that generation 2 RCE was to be replaced, with a tender to be called after a business case was approved by the TasRail board.
In February 2017, a project manager for the RCE replacement project was appointed. TasRail also contacted ADE again to discuss the RCE replacement. At this time, on TasRail’s request, ADE provided an updated quote, again for RCE units sourced externally. TasRail’s project manager wrote to a colleague that, due to continuing delays in the project approval process, TasRail would need to start the process of writing to ACMA to ask for an extension for the radio spectrum allocation.
The RCE replacement proposal was approved at executive level in April 2017. In mid-May 2017, a meeting was held between the TasRail RCE project team and ADE to discuss TasRail’s technical requirements for generation 3 RCE, including:
retaining compatibility with both DQ and TR class locomotives
compatibility with the new radio spectrum using a digital radio (previously analogue)
inclusion of the Railton loading interlock (also on generation 2 RCE)
addition of dynamic brake
exclusion of the GPS speed restriction option.
ADE advised TasRail at this meeting that the externally-sourced RCE was no longer an option as it could not accommodate TasRail’s unique requirements of the Railton loading interlock, dynamic brake and radio frequency change. The externally-sourced RCE was also primarily designed for marshalling yard work, not main-line operations.
Instead, ADE advised that it would manufacture and supply TasRail with the generation 3 RCE. This process ultimately resulted in an almost entire redesign, in part due to the requested additional functions and the need to source replacements for the microprocessors.
In late May 2017, TasRail ordered two generation 3 RCE units from ADE, with the addition of dynamic brake capability. Correspondence between TasRail and ADE during the remainder of the month included:
driver feedback on transmitter layout
addition of high and low horn functionality
addition of an aerial on the driver’s van, including an accompanying external port and selection switch on the transmitter
radiofrequency spectrum allocation for the RCE.
Although delivery of generation 3 RCE was scheduled for October 2017, several issues delayed delivery. These issues related to last-minute radio frequency changes, discontinuation of antenna manufacture by the supplier, and the RCE intermittently failing-to-safe during bench testing.
These delays required driver training and RCE commissioning testing (which coincided with limited cement train shutdown opportunities) to be rescheduled. Frequent extension requests for the existing radiofrequency spectrum allocation by TasRail to emergency services were also required to allow continued use of the generation 2 RCE.
TasRail’s initial project manager left in November 2017 and was replaced in mid-December by another project manager. This (final) project manager told ADE in December 2017 that continued delays were ‘getting out of control’ and that further slips were unacceptable.
The generation 3 RCE was ultimately delivered for commissioning testing on 18 January 2018.
Neither TasRail or ADE provided the ATSB with a comprehensive list of changes between generation 2 and generation 3 RCE. However, a review of evidence by the ATSB indicated that the TasRail generation 3 RCE project team were aware of several changes between the RCE generations. These included:
new processors and motherboards (which also required the software to be rewritten in a new language in both the transmitter and receiver)
conversion from analogue to digital radio on a new frequency
addition of dynamic brake
addition of high / low horn functionality
addition of an aerial on the driver’s van, including accompanying external port and selection switch on the transmitter.
Comparison between the generation 2 and generation 3 RCE operation manuals and schematics, review of correspondence, and interview evidence established that there were about 40 changes between the two generations of RCE. Records indicated that there were many undocumented changes, including:
control fault mode receiver response
communication failed mode receiver response (automatic and independent brake change)
functionality of ‘bail-off’ control
removal of data logger from the receiver
new modes of operation and initialisation sequence
alteration to transmitter audible alerts
alterations to both transmitter and receiver displays
locomotive vigilance system suppression and RCE receiver penalty response
removal or change of several states that invoked an ‘emergency’ mode receiver response
reversal of switch orientation to command an emergency stop
alteration of receiver response when direction was set to neutral
brake pipe pressure reduction levels in relation to automatic brake lever positions
alteration of reverser violation penalty (changing direction while traction power applied) from an emergency brake application to power-down of applied traction power only.
Development process and documentation
TasRail had formed, but not formalised, a conceptual view of how the RCE would operate in practice. That is, that a system would be installed on the TR class locomotive that enabled a single driver to drive the train from within the locomotive, from the driver’s van, or while loading from outside the train. There was also an identified need for certain high-level functions, such as the Railton loading interlock.
This information was communicated to ADE prior to the completion of the generation 2 RCE, while the need for additional high-level functions were advised to ADE during meetings in 2017 regarding the generation 3 RCE. After May 2017, according to several TasRail managers, TasRail had virtually no direct involvement in the generation 3 RCE design process until the commissioning testing in January 2018. ADE reported that there were no regular meetings for the duration of the generation 3 design project.
TasRail did not provide ADE with a documented set of requirements for the generation 3 RCE, including safety requirements, and it did not provide operational context and risk profile information that could be used to generate safety requirements. There was no explicit set of requirements or information about hazards and risks being documented by either TasRail or ADE.
Although some engineering design documents relating to the locomotive and braking systems had been sent to ADE by TasRail, ADE reported that it did not receive sufficient documented system information for development, and needed to make some assumptions about systems and operations and were reluctant to ask for design documents due to intellectual property concerns.
ADE obtained an electronic brake control system manual (similar to that used for the TR class locomotive) from the internet. It did not consult directly with the train manufacturer or the electronic braking system manufacturer. Although the content of this manual described a similar system to that implemented on the TR class locomotive, some nuances of programmable functions were not described (for example, brake cylinder pressure, as affected by the J-multiplier). Further, although functionality of the EMV was described, there was no reference to the VX vent valves’ existence or operation.
All ADE’s physical and electrical design work and management of the project were performed by one person. Checks (reviews) were recorded for some design drawings, but these were by the same person.
According to ADE, the software was redesigned for the generation 3 RCE by a software engineer engaged specifically for this purpose. ADE did not retain permanent in-house capability to maintain the software and engaged the software engineer as needed.
The design documents produced by ADE for the generation 3 RCE are listed in Table 4.
Table 4: List of generation 3 RCE design documents
Document(s)
Date(s)
Delivered to TasRail
Operation manual
December 2017, revised July 2018
Yes
Physical specification (shape and dimensions)
July 2017
Yes
Electrical schematics
August 2017 – January 2018
No
Pneumatic schematics
January 2018
Yes
Pre-delivery and commissioning testing report
January 2018
Yes
ADE did not provide evidence to TasRail or the ATSB to show that the generation 3 RCE met any systems or safety engineering standards, or that the extensive process, analysis, technical and documentation requirements of a formal system safety process (such as outlined in AS 61508) were followed.
Driver training
With the introduction of generation 3 RCE, ADE provided additional training to drivers and supplied each driver with a copy of the RCE operation manual. The training consisted of a 20-minute session and was provided on a test bench at the RCE maintenance contractor’s premises in Devonport, with the assessment consisting of a checklist. The driver of train no. 604 undertook this training along with 3 other drivers on 16 January 2018, 3 weeks before the introduction of generation 3 RCE.
Of the 40 changes identified by the ATSB between the two RCE generations, many were not recorded as having been assessed, including:
functionality of ‘bail-off’ (see Automatic brake)
transmitter audible alerts
operation of dynamic brake
vigilance suppression and RCE receiver penalty response
brake pipe pressure reductions in relation to lever positions
reversal of switch orientation to command an emergency stop
significant changes to communication failed mode and control fault mode RCE receiver response (see also Changes to braking effort during communication failed mode).
One change that was assessed was related to the direction controller. The assessed item was in relation to the ‘reverser violation’ function. However, this was different and unrelated to the dual direction fault interlock, which was not assessed during training.
Commissioning
Pre-delivery testing (sometimes termed factory acceptance testing within industry) of both generation 3 RCE units was conducted at ADE’s workshop using a test bench on 7 January 2018. The testing report listed 13 test elements conducted on both of the RCE units that ADE delivered to TasRail (Figure 15). There was no recorded test procedure or other records to describe the purpose, method, criteria, or outcomes of each test. All tests were marked with a symbol that denoted ‘a validation for the test’; however, no relevant validation records were available.
Figure 15: Generation 3 RCE pre-delivery test record
Source: Air Digital Engineering
On 18 January 2018, with ADE in attendance, generation 3 RCE was installed on the cement train for the first time to undergo commissioning testing. A single return trip from Devonport to Railton with one of the RCE units was conducted, which included the use of dynamic brake and loading and unloading movements. During this testing, ADE found ‘…a requirement for an adjustment in this remote unit’ necessitating the return of the 2 RCE receivers to ADE’s workshop in Sydney. There was no record made of how the tests were conducted, the exact intent of the tests, or the outcomes.
ATSB assessment of this activity indicated that the tests conducted by ADE showed only that the RCE was capable of performing certain functions in a general sense. In the absence of test procedures or other documentation, there was no way to show whether any non‑trivial safety objectives were met. For example, there were no records to show that functions would operate under all operational conditions, including different train configurations.
In addition, there was no record made of what the subsequent remedial repairs were, or if the pre-delivery testing was again performed on ADE’s test bench. ADE advised TasRail that, on return of the RCE to Tasmania, it would ‘…need to be properly tested again, preferably on a TR locomotive before putting it back on the Cement train’.
On 6 February 2018, commissioning tests of the two generation 3 RCE units were again undertaken. A test plan for this was developed by the TasRail project manager and reviewed by ADE, the rolling stock assets manager and driver supervisor. The plan was developed from technical advice supplied by ADE and the RCE operation manual.
The tests were conducted in the presence of ADE and the RCE maintenance contractor, with supervision on behalf of TasRail by a graduate engineer (who had started with TasRail the previous month) and some operational staff. The graduate engineer advised the ATSB that their first interaction with the RCE was on the day of the tests; they had very limited knowledge of the TR class locomotive systems, and their role was limited to observing the testing conducted by ADE. Engineering approval authority rested with the supervising engineer (the rolling stock assets manager), who was not present.
The testing was performed in Devonport Yard with limited train movement available. The test plan (a checklist, Figure 16) included items such as operational functionality, transmitter LED indicators and safety features.
The checklist was initialled by the graduate engineer to indicate that the test was conducted (initials obscured). Source: TasRail, modified by the ATSB.
The commissioning checklist did not denote defined steps, expected observations or pass / fail criteria. Further, it did not include several failure modes, which aligned with tests 8 through 13 on the pre-delivery testing document. These tests were labelled on the pre-delivery test document as: ‘Test requires embedded circuitry modification to verify correct response for the failure’. This indicated that these tests were only able to be conducted on the ADE test bench.
Over the following 3 days, additional tests related to this list were conducted on the cement train service between Devonport and Railton, with ADE in attendance. The TasRail graduate engineer initialled next to tests 1–14 and 17–25 in the first table (Figure 16), and in the header block above the second table. These indicated which tests had been performed. There were no further records of how the tests were performed, or the results.
Both generation 3 RCE units were formally accepted by TasRail as being ‘…commissioned successfully, without error and only minor omissions…’ on 9 February 2018. There was no record of what the ‘minor omissions’ were.
Previously, in early 2014, TasRail undertook a substantial change to locomotive brake systems[56] that required locomotive and RCE testing. Like the generation 3 RCE test records, there was a list of tested functions (Figure 17) but no test procedure. By contrast, an independent consulting firm engaged by TasRail to verify the proposed locomotive functionality had a 16-page test procedure with defined steps, expected observations and detailed pass/fail criteria. An example procedure (Figure 18) shows how the consulting firm detailed how it tested the same functionality listed as item (k) in the ADE record.
Figure 17: Extract of RCE test documentation used by ADE in 2014
Source: TasRail; unrelated highlighting removed by ATSB
Figure 18: Extract of locomotive test procedure used by independent consulting firm in 2014
Source: TasRail
The generation 3 RCEs were used on the cement train service immediately following the 9 February 2018 commissioning tests. Once the generation 3 RCE entered service, ADE personnel rode with drivers for a short period of time to monitor RCE operation and performance.
Issues after commissioning
Screen freezes and radio signal strength
On 11 February 2018, problems with the RCE transmitter display screen were encountered. The transmitter was found to be frequently locking up (with the screen ‘freezing’) with no updates of air pressures or operating status available to the driver. ADE advised TasRail that this was a ‘…nuisance occurrence but…drivers are knowledgeable and using the Led’s’; that is, ADE believed operations could continue based on the transmitter’s LED indications alone.[57] Driver reports indicated that these screen freezes were frequently in conjunction with slow or non-responsiveness of the RCE to driver commands and communication failed mode occurrences.
In response, ADE communicated with the cement train drivers by telephone over a 2-day period (from 17 to 18 February 2018) to diagnose the fault. ADE initially believed the problem was caused by excessive radio signal strength between the transmitter and receiver causing transmitter-screen interference. The radio signal was altered to a lower strength on several occasions, however this did not resolve the transmitter lock-up faults.
At the end of June 2018, with work continuing on addressing reported problems, ADE found that the transmitter lock-ups were being caused by an ‘…incorrect reference to a microprocessor setting in the manufacturers information’, not radio signal strength. Adjustments were made and the original radio signal strength was restored. These changes greatly improved the RCE’s reliability and performance from late-June 2018 onward, including a reduction in communication failed mode events.
Traction motor flashover event
On 13 February 2018, a traction motor flashover event[58] occurred, damaging a traction motor on a TR class locomotive. After investigation by TasRail, the locomotive manufacturer and ADE, it was found that, while travelling in the forward direction, the driver had accidentally moved the direction controller from forward to reverse instead of de-selecting dynamic brake on the transmitter. Later, after de-selecting dynamic brake correctly, the driver applied traction power while still travelling forward, resulting in a wheel lock and flashover.
TasRail identified that the dynamic brake selection switch and direction controller looked and operated the same and were adjacent to each other on the transmitter (see Figure 13). In response, the use of dynamic brake on the cement train was suspended until changes were made to the RCE by ADE to prevent recurrence (see Changes to remote control equipment prior to project completion). Although the changes were made by June 2018, the drivers were not advised. As such, at the time of the 21 September 2018 runaway accident, dynamic brake was still not in use.
Changes to braking effort during communication failed mode
As discussed in Differences from previous generation, among the changes between the generation 2 and generation 3 RCE were changes to the communication failed mode response. The generation 3 RCE now applied a lessened automatic brake pipe pressure reduction (400 kPa instead of the previous full-service application of 350 kPa) in communication failed mode, and also maintained any previous independent brake application.
Although it is possible that the matter was discussed without being recorded, and the RCE operation manual had reflected the new response, the available evidence indicates that TasRail did not initially consider the potential effect of these changes on train stopping distance in communication failed mode. In April 2018, 2 months after the introduction of generation 3 RCE into service, the network access manager became aware of the changes and enquired about whether the change had been risk assessed.
TasRail then asked ADE if this change had been in the original scope of work. ADE advised that the change had been brought to the attention of the TasRail generation 3 RCE project team. This appeared to have occurred in an October 2017 email to TasRail (at the end of a list of unrelated questions) where ADE mentioned a change to the independent brake response for the generation 3 RCE during a communication failed mode response:
…if there is a communication loss, I would like to be sure that by applying the safety factors for slower train speeds will also be suitable when the train is moving fast.
With the new systems, as well as the Brake Pipe application the independent brake will maintain whatever the output command was before the comms loss [communication failure].
In any case, neither ADE or TasRail assessed the safety implications of the reduction in automatic braking level at this time. It is therefore likely that TasRail, at the time, did not appreciate that the train’s braking performance in communication failed mode would be decreased with the generation 3 RCE compared to the generation 2 RCE.
Stopping distance tests were ultimately performed by TasRail in mid-June 2018, at typical yard speeds (up to 25 km/h). There were difficulties with recording fidelity, and later ATSB analysis concluded that the available results were inconsistent. Furthermore, although TasRail concluded that the new response ‘sufficiently brings the train to a stop’, there was no risk assessment or other analysis to determine the safety impact of the change. TasRail later advised the ATSB that the testing was intended to address the potential for loss of radio communications during yard operations, where the driver would be external to the train, and considered that driver access to the emergency dump valve in the locomotive would enable more rapid stopping in an emergency (while driving at speed) than the application of the communication failed mode braking response.
An action item that was generated as a result of this testing indicated that further driver training or instruction was to be provided on the continued availability of the emergency brake during a loss of RCE radio connection through operation of either the locomotive automatic brake handle or driver’s van emergency brake (see Emergency brake). Although this training was noted as completed on 1 June 2018, available evidence indicates that it did not occur.
Other problems
Throughout the period from February to June 2018, drivers were reporting a significant number of fault events associated with the generation 3 RCE, primarily communication failed mode events as well as transmitter lock-up events (see Reported faults).
Further problems identified after commissioning of generation 3 RCE included:
There were 2 different types of multiple unit cable pins within the receptacles across the TR class locomotive fleet. Identified in late February 2018, this resulted in ill-fitment with the receiver’s multiple unit plug, causing issues with the selection of forward and reverse directions.[59]
There was an uncommanded brake release and risk of runaway at Railton when resetting a communication failed mode occurrence. First identified in early May 2018 after a software change, it was found necessary to apply a full-service automatic brake application on the transmitter during each wagon placement at Railton to prevent an uncommanded brake release in the event of a communication failed mode reset. ADE advised TasRail in late May 2018 that the issue had been fixed through a further software change, ensuring the brake pipe would not recharge when the RCE transmitter was switched back to ‘interlock’ from the ‘off’ position. As a result, drivers were instructed that a full-service automatic brake application after wagon placement was no longer required. However, less than 4 hours later, after a further undisclosed RCE fault was encountered,[60] the full-service instruction was reimposed. It appeared this issue was resolved at the end of May 2018.
On 25 May 2018, a failure of the RCE transmitter’s throttle occurred while tractive power was applied. The driver reported that they had used the TR class locomotive emergency dump valve to stop their train after being unable to remove tractive effort on the transmitter. TasRail advised ADE that the driver did not use the transmitter’s emergency stop switch as ‘The driver did not trust the remote [transmitter]’. TasRail further advised ADE that ‘The remote [RCE] incidents have become a safety issue and require immediate rectification’. Another failure of the transmitter’s throttle had also occurred 6 days earlier, with repairs made by the RCE maintenance contractor.
Issues were reportedly encountered after fault mode resets. These related to an inability to release the independent and / or automatic brake, or an inability to reapply the automatic brake after fault resets.
TasRail response
In mid-May 2018, TasRail expressed to ADE concerns of a ‘significant risk to the business’ due to the ongoing generation 3 RCE faults, and it requested ADE to supervise the next onsite RCE changeover[61] to ensure ADE’s latest modifications were successful. ADE did so and was presented by TasRail with a list of faults to address. Two follow-up meetings were held between TasRail and ADE in June 2018 to address these matters. TasRail later advised the ATSB that, at this time, it was concerned with the communication and braking issues.
TasRail expressed to ADE twice (17 May 2018 and 6 June 2018) that it intended to reinstate generation 2 RCE until such time as ADE had rectified the ongoing faults with generation 3 RCE. However, this did not occur. Safety was not explicitly stated among TasRail’s concerns. As a TasRail representative described to the ATSB, TasRail’s concerns were from operational and business perspectives due to time delay impacts. The decision was made to continue using generation 3 RCE as:
A large number of communication failed mode events had already been occurring with the end-of-life generation 2 RCE.[62]
There was a need to test fault rectification modifications to the RCE by ADE.
Although RCE drivers had raised safety concerns regarding generation 3 RCE, ADE gave TasRail assurances that the RCE would fail safe.[63]
By late June 2018, the RCE’s reliability had considerably improved. Reporting of communication failed mode events (most reports) had reduced to an ‘acceptable level’.[64]
By the time of the June 2018 meeting, most RCE problems were occurring during loading and unloading activities.
Changes to remote control equipment prior to project completion
After the traction motor flashover event in February 2018, ADE altered the dynamic brake selection switch from a forwards / backwards to a left / right orientation, to differentiate the switch physically from the direction controller. In addition, ADE took the opportunity to add time delays before the RCE receiver would apply traction power after de-selection of dynamic brake, and before the application of dynamic braking effort after selecting dynamic brake mode. Both these changes were made by late June 2018.
Other changes to generation 3 RCE after commissioning, but prior to acceptance, included:
multiple software changes
communication failed mode behaviour while in Railton loading interlock mode
alteration of ‘bail-off’ capabilities
a reduction in radio signal strength, which was subsequently reverted
radio component changes
change in level of brake pipe pressure reduction for a minimum-service application.
Most of the changes above were made without any record of subsequent testing. On 2 occasions, TasRail recorded that testing had occurred; however, there was no description of the tests conducted or the outcome of testing. Both occurred in June 2018. The first test was in response to several modifications that had been made to improve reliability of the RCE. There was no record made of what the changes were. The second test was of the impact to cement train stopping distances due to changes to the communication failed mode, as discussed previously in Changes to braking effort during communication failed mode. Although this change existed in the commissioned version of generation 3 RCE, it was not tested at that time.
No risk assessments were performed for any of the changes to generation 3 RCE after commissioning (see also Risk assessments for remotely-controlled train operations).
Acceptance and project completion
The post-commissioning modifications to the generation 3 RCE from late June 2018 resulted in greatly improved reliability. Fault reports had reduced from up to 20 per day earlier in the month to 1 or 2 per shift. As this was deemed to be an acceptable level of faults by both ADE and TasRail, generation 3 RCE was accepted for practical completion on 26 June 2018.
Asset handover from the RCE project team followed on 2 August 2018, noting that dual inspection by both the asset manager and end user was yet to occur. The project was closed with the following comments:
The new remote controls units have had over the past six months undergone Extensive testing, modifications both on and off the cement train [and] has provided a Safe and efficient remote control mechanism that is both compliant with ACMA, TasRail and ONRSR [Office of the National Rail Safety Regulator] requirements.
The periodic loss of communications with Communication failures (CMF) [communication failed mode] has been adjusted and is currently held to a manageable minimum and will continue to be monitored to identify any trend or location issues with the remote areas.
Remote control equipment fault reporting
Fault reporting process
There was no dedicated form or procedure for reporting generation 3 RCE faults, although drivers could report faults through other means. For example, driver reports received by the duty network control officer (NCO) would be noted in the NCO incident log. Accordingly, reports of substantial time delays attributable to the RCE were recorded in the log.
In addition, drivers completed a cement train shift form each day during their shift. This form included multiple options for recording time delays, with one termed ‘remote issues’. This form included both reports that were also made to the NCO, and further faults and events which did not result in significant time delays.
Driver reports received by the duty NCO would be referred to the relevant section for action. For generation 3 RCE faults, these were to the driver supervisor, rolling stock assets manager or superintendent technology and communications. However, the decision of who was best placed to address these was at the NCO’s discretion. In addition, the customer service delivery manager would tabulate RCE specific cement train shift report delays and forward these to the TasRail project manager, who would then forward a selection of these to ADE for diagnosis and rectification.
Reported faults
From 9 February 2018 (the date that generation 3 RCE was commissioned) to 21 September 2018, there were 68 entries for a total of 125 faults identified in the NCO incident log and 116 entries totalling 281 faults identified in the collated cement train shift reports.
It is likely that many of the entries in these 2 types of forms were duplicates (the same occurrence being recorded by both reporting channels). However, there was insufficient information to confidently identify all duplicate entries, and some entries that were likely to be duplicates recorded different fault conditions for potentially the same fault. Therefore, no attempt has been made to remove duplication from the analysis presented below. There was also insufficient data available to determine the validity of the reported fault type with confidence.
Acknowledging these limitations, the ATSB analysis indicated that TasRail recorded:
214 communication failed mode events
90 events where the fault mode was either not specified or not a penalty mode (for example, throttle failure, Railton loading interlock failure and brakes not releasing)
17 driver-initiated receiver shutdown and restarts (cold starts) to reset fault conditions
25 uncommanded emergency brake applications or alarms
4 control fault mode events
1 driver reporting the RCE as ‘unsafe’ for use
1 driver ceasing use of the RCE after it became unresponsive.[65]
Some of the reported events, such as the cold starts, almost always happened because of other faults (as they were recovery actions), and others were reported simultaneously (such as simultaneous communication failed mode and transmitter freeze).
Although on most days there was only a single fault recorded, this varied. Occasionally reports included over 10 faults in a single day, with the most recorded being 20 in one day in June 2018. As previously noted, the number of faults significantly decreased after June 2018, down to 1 or 2 per shift.
Extent and accuracy of fault reporting
A TasRail representative expressed concerns in interview to the ATSB that drivers had become overly confident with RCE failure resets with the generation 3 equipment, and were under-reporting faults. The representative stated drivers thought if ‘I do a reset and it keeps running, I’ll use it…until it stops’. These thoughts were shared by the incoming driver (on the day of the accident), who advised the ATSB that, provided an RCE fault reset was successful, operations would continue. TasRail drivers also advised that they would only log and report generation 3 RCE faults with their supervisor or the NCO if the faults were persistent.
In May 2018, it was recognised that not all RCE faults were being reported by drivers, including some being reported informally to the RCE maintenance contractor. This resulted in the driver supervisor sending an email to all RCE drivers, requesting urgent advice of all encountered faults ‘…no matter how trivial’, to coincide with an onsite attendance by ADE for rectification.
Concerns regarding the informal nature of fault reporting by drivers was again raised at a meeting between the generation 3 RCE project team and ADE in June 2018. However, the issue was not formally recorded on the meeting minutes, nor noted who raised the concern. No evidence was provided by TasRail to indicate how, or if, this problem was addressed.
The seriousness of control fault mode events was recognised by the network access manager, who had previously implemented generation 1 RCE in their former role as the planning manager. In interview, the network access manager described instances of control fault mode as an unsafe condition that required an emergency brake application by the RCE because it was quite serious. They also noted that, if the locomotive was moving, the emergency brake application would create an alarm at network control.[66]
The network access manager reported that they had reviewed driver reports and noted that several network control centre emergency alarms with generation 3 RCE were being logged as communication failed mode events. They stated that this was incorrect as these did not send a Selcall alarm. They also noted that communication failed mode events were, in their experience, viewed as a productivity issue not a safety issue.
Furthermore, it was common in their experience for communication failed mode and control fault mode events to be misdiagnosed by drivers as they looked similar on the RCE transmitter display screen (that is, they had similar letters, with communication failed indicated by ‘COMF’ and control fault indicated by ‘CNF’, Figure 19). This view was shared by the NCO on shift at the time of the 21 September 2018 runaway accident, who advised that it was normal for a communication failed mode to trigger an emergency alarm. The NCO would then follow up and the driver would also indicate that it was a communication failed mode event.
Figure 19: RCE transmitter and display screen
Image shows the RCE transmitter with detail images of communication failed mode (left) and control fault mode (right) presentations on the transmitter screen. Source: ATSB
To ensure appropriate scrutiny and tracking of control fault mode occurrences within the driver reports, the network access manager advised the NCOs to challenge drivers who reported communication failed mode occurrences when there was an accompanying emergency alarm in the control centre.
Both the network access manager and NCO reported having an expectation that communication failed mode or control fault mode events would create Selcall alarms. They also received reports of these events after receiving Selcall alarms. However, this was not the designed behaviour of the generation 3 RCE, which were not intended to create an alarm. The control fault mode response had been changed from the generation 2 RCE (which previously would create an alarm when moving) and the communication failed mode response also was not intended to create an alarm.[67]
As discussed in Other selected fault events, there were at least 8 instances of uncommanded emergency brake applications reported by drivers. Although it is likely these were considered to be instances of control fault mode events, the ATSB found that none of these events were related to either a known fault mode or any other emergency penalty condition.
In addition, during review of the NCO incident log records, the ATSB identified at least 2 occasions where a driver had reported a full-service penalty, as occurs in control fault mode, as a communication failed mode event.
In summary, there was evidence of incorrect expectations, misreadings and mode confusion in reports made by the generation 3 RCE drivers and received by network control. This resulted in periodic misdiagnoses of control fault mode (CNF) events with communications failure mode (COMF) events. In addition, uncommanded emergency brake applications, which were not attributable to any known fault or penalty condition, were at times misdiagnosed and reported by drivers as either communication failed or control fault mode events.
TasRail response to reported faults
In instances where driver reports received by the NCO were deemed to be a risk to safety (including ‘Category A and B’ notifiable occurrences), the NCO was to create an entry in TasRail’s ‘risk wizard’ system. This incident recording database allowed for the identification, tracking and recording of actions against identified safety incidents. From the introduction of generation 3 RCE to the day of the accident, only one entry in relation to the RCE was made in risk wizard. This related to the event at Caroline Creek in May 2018 (see Caroline Creek event), and was recorded on the basis of work health and safety considerations due to lack of trackside access to the locomotive. This single risk wizard entry for generation 3 RCE did not include any follow-up actions (see also Occurrence notification).
As noted above, RCE driver fault reports received by the network control centre were referred by the NCOs for follow-up. Communication failed mode occurrences were occasionally sent for follow-up, although this appeared to be dependent on which NCO was on duty at the time of the occurrence. The rolling stock assets manager was also frequently advised of faults.
The rolling stock assets manager advised the ATSB that, after the commissioning of generation 3 RCE, they worked with ADE (through the project team) to rectify rolling stock–RCE integration faults, including the issues with dynamic brake, multiple unit pins and communication failed mode events (see Remote control equipment and Issues after commissioning). The rolling stock assets manager would forward RCE specific faults to the RCE maintenance contractor for rectification. Once final acceptance of generation 3 RCE was complete, the rolling stock assets manager worked directly with ADE.
Several TasRail staff advised the ATSB that they had been concerned regarding the overall monitoring, coordination and follow-up of generation 3 RCE faults. There was a heavy reliance on ADE and the RCE maintenance contractor for a technology that was largely an ‘unknown’ within TasRail.
The ATSB was advised these concerns were raised at the daily TasRail operations meetings; however, there were no records available to confirm this had occurred.
TasRail change management and related processes
General information
TasRail was required to have procedures within its safety management system (SMS) for ensuring that changes that could affect the safety of its railway operations were identified and managed.[68] This included ensuring that changes were fully identified and described, and the risks associated with the change were identified and managed.
The TasRail change management procedure (BIC-PRO-300) detailed that, other than ‘routine change’,[69] changes were to include a:
consultation process
change management plan
risk assessment
documented record of the changes
check of compliance against legislation
safety validation where the change impacted ‘safety activities’ (with safety activities including ‘driving and operation of trains’ and ‘control the movement of trains’).
Change management processes
According to the change management procedure, all changes were to be classified by an initial impact assessment (IIA) to determine the project management level required. The IIA was split into 2 parts:
Part A included 9 questions, answers to which determined if the change was to be considered ‘small’ (checklist only) or ‘significant’ on the basis of factors such as:
impacts on people, processes, or systems
inter-departmental cooperation
safety (‘Are there any negative safety implications associated with the initiative?’)
strategic influence
cost
requirement to notify ONRSR in accordance with notification of change guidance.
Part B applied to changes that had been assessed as significant. This part asked 6 questions on business and safety risks, some of which were broken down into more detail. Each question could provide up to 5 points, and the points were totalled to give the project a ranking of P0, P1, P2 or P3 in increasing order. Less than 14 points resulted in a classification of P0 or P1 depending on cost, while 14-21 points resulted in a classification of P2. P0, P1 and P2 projects were further ranked with a ‘+’ if the project had added ‘significant’ strategic or ‘high’ safety impacts.
At the end of the IIA form, there was a ‘project management assurance requirements and approvals record’, listing several ‘gates’ (or checkpoints) for P1 and P2 projects, as follows (in addition to provision for other gates as required):
approval of the project classification
acceptance in principle (P2 only)
approval to proceed
approval of the project business case
acceptance of design
acceptance for testing (P2 only)
acceptance for operations/implementation (P2 only).
All these gates required authority from the executive to proceed. There was also a project closure checklist required to be authorised by the project sponsor.
All ‘significant’ projects required a range of activities, such as risk assessment, risk action planning, life cycle safety validation, and reviews, each producing documentary records. Projects with higher classifications required additional approvals and documents.
Two of the defined processes for significant changes were BIC-PRO-302 (Change management processes for P0/P0+/P1/P1+ projects) and BIC-PRO-301 (Change management processes for P2/P2+/P3 projects). Until late July 2017 (during the generation 3 RCE project), both were in draft form and not available on TasRail’s intranet. However, TasRail advised the draft processes were ‘…possibly used in-house’ during this period. Documents produced for the project indicated that this was the case.
Both these processes contained a series of ‘gates’, which served as checkpoints that required documentation and / or approvals before the project could progress. The project management processes contained the regulatory requirements for change management (see Notification of change).
The change management procedure and processes (the latter of which were more related to project management) mentioned the requirements for conducting technical investigations, creating preliminary designs, and performing safety validation for the whole of the system life cycle for the assessed change. However, there was no direction to the relevant sections of TasRail’s SMS or external guidance materials that would apply in this regard (for example, relevant engineering standards, procedures and processes, see Design assurance processes for further information). Such guidance was particularly relevant for accurately assessing the safety impacts the change may have on current operations and conducting appropriate activities to address these impacts during the change process.
Generation 3 remote control equipment project
Business case
In early 2015, the generation 3 RCE project informally commenced (see Generation 3 remote control equipment project initiation). A formal project business case was completed in December 2015. It noted a requirement to replace the ageing generation 2 RCE due to:
reduced reliability
end of ADE support
lack of dynamic brake capability
inability to transition the current RCE to a new digital radio.
Four options were presented in the business case, including a ‘…simple like for like replacement’ of the generation 2 RCE units, with:
the addition of dynamic brake capability
compatibility with the newly-allocated radiofrequency spectrum allocation.
Initial impact assessment
An initial impact assessment (IIA) for generation 3 RCE was completed in November 2016. The completed IIA referred to the addition of dynamic brake but did not include consideration of other changes to generation 3 RCE that were known or requested by TasRail. It did not specifically identify the dynamic brake as a change from the previous generation of RCE, just that it was included.
Part A of the IIA was completed as follows:
The summary of the main change implications of the initiative was ‘continued use of remote control systems in a safe manner and in compliance with regulations’.
There were ‘Yes’ answers to the questions about impacts on people, processes or systems, inter-departmental cooperation, and cost.
There was a ‘No’ answer in relation to whether there were ‘any negative safety implications associated with the initiative’.
There was a ‘No’ answer in relation to whether notification of change to ONRSR was required, with a note that read: ‘…assuming that as a replacement part units can function under any authority currently granted……need to confirm’.
The question to record whether the change was significant was not completed, but Part B was completed, indicating that the ‘Yes’ answer in Part A led the assessor to determine that the change was significant.
Items of note in Part B included:
The assessment of risk (to business, people or processes) was ‘low’. The guidance description for the low-risk category was that ‘initiative is an administrative change or a change of like for like’ and the associated justification was that the proposed RCE was the ‘same units which are more modern but perform the same tasks more safely’.
‘Professional human factors services’ were not required.
In determining the ‘failure consequence’, which asked ‘Impact due to worst case scenario arising from system failure or human error?’, the assessor recorded the middle score (3 points), associated with ‘limited…safety, operational or organisational impact’ (the other options were ‘minimal’ with 1 point or ‘significant’ with 5 points).
In response to an ‘assessment of complexity,’ the assessor recorded that there was ‘minimal systems integration’ and ‘straightforward’ incorporation of safety requirements. The overall assessment of complexity, based on a combination of other assessments, was the middle of 3 available scores.
In response to an ‘assessment of novelty,’ generation 3 RCE was stated as ‘in widespread use’ outside of TasRail, and that the implementation did not require new standards or procedures. The overall assessment of novelty, based on a combination of other assessments, was the lowest of 3 available scores.
The assessment of strategic value was ‘None’. The guidance description for this category was ‘initiative is a change of like for like and has no impact on TasRail’s business value.’
The assessment of impact on safety was ‘Medium’. The guidance description for this category was ‘initiative is an adaptation of an existing system, function or asset’.
The overall score of 12 resulted in the change being classified as a significant change of level P1 (minor business impact, project value outside general manager’s financial delegation) requiring the ‘significant change management process’ to occur.
The initial project manager reported that the IIA was checked by the project stakeholders identified within the IIA and by the general manager of asset services. However, the project management assurance requirements and approvals record was blank and TasRail had no evidence that the necessary approvals had been obtained.
Revised business case
In January 2017, the business case for generation 3 RCE was resubmitted. As with the business case about a year earlier, it recommended the purchase of two RCE units with dynamic brake capability, noting it was a ‘simple like for like replacement’, although training would be required. Known business and operational risks of not proceeding with the purchase were detailed, with the risk register flagged as requiring updating.
In the ‘change management plan’ section of the business case, the project level was increased to P2 (important business impact) from the previous IIA-assessed P1 level. The relevant procedure stated that where an increase in scope or cost of the project occurred, as can be inferred from the increase to P2 level, another IIA was to be completed and submitted for executive approval. However, TasRail could not provide a reworked IIA or other details explaining why the project level had been increased to P2.
Although termed a ‘like for like’ replacement, the person who developed the business case told the ATSB that the inclusion of the options represented a ‘significant change’ that affected the assessment, and that, in hindsight, these changes meant that it was not like-for-like.
The business case recommended single-source supply procurement from ADE, rather than a tendering process, as ADE was:
familiar with TasRail’s operational requirements (having previously supplied generation 1 and generation 2 RCE)
able to assist TasRail with developing the required technical specifications
available for local support
able to meet the ‘tight timeframe’ for delivery of generation 3 RCE by May 2017, with commissioning complete by the end of August 2017 (due to ACMA requirements).
The business case on multiple occasions referred to adhering to the project management process as a means of fulfilling the business case requirements.
As noted in Generation 3 remote control equipment project initiation, TasRail appointed a project manager for the project in February 2017. TasRail also contacted ADE to obtain an updated quote for the RCE and proposal was supported at executive level in April 2017.
TasRail’s risk and compliance manager advised the ATSB that, during this period, they were aware that an upgrade of the RCE was occurring; however, they were not involved in the project as it had been deemed ‘like for like’. Neither the freight services or assets management safety specialists (who reported to separate general managers) were involved during the generation 3 RCE procurement and commissioning process.
Change management activities
TasRail’s change management processes listed gates (3 for a P1 project and 6 for a P2 project) intended to prevent a project from progressing unless certain requirements were met. In addition, the processes listed 20 documents required to be produced for P1 projects, and 33 documents for P2 projects.
TasRail produced 11 of the listed project documents. Of these, 5 were incomplete, including 3 not containing the required approvals. There were no records to show that the 6 required gates for a P2 project were formally passed with the appropriate executive-level approvals.
Relevant documents required for a P2 project but not produced included:
risk assessments
notification of change process
change management plan
stakeholder engagement and communications
technical investigations / preliminary designs
regular monthly reports to the TasRail Board
acceptance for testing form
acceptance for operations form
post project review report.
TasRail’s internal investigation report into the 21 September 2018 runaway accident found that:
Based on the long safety record of previous generations [of RCE] and in the belief by the TasRail project staff that the third generation remote-control was an enhanced “like-for-like” and fail-safe replacement, TasRail’s procurement processes were truncated...No specification was issued and the procurement was treated as a product purchase. As a result, TasRail’s normal project management and engineering controls were circumvented and full reliance placed on the supplier to deliver and commission the units.
As noted above, the concept of the generation 3 RCE being a like-for-like replacement was used in the December 2015 business case and the initial impact assessment (IIA). The initial project manager also advised the ATSB that it was a like-for-like replacement that replaced one remote with another, and that related safety procedures were still valid for the new RCE, excepting those relating to the new dynamic brake feature. They therefore felt that the safety impact of the replacement was low. In addition, TasRail advised the ATSB that the limited amount of driver training required indicated that the operational changes were not substantial, reinforcing the perception that the generation 3 RCE was comparable to the previous generation.
Even though there was a view within the project team that generation 3 RCE was a like-for-like replacement product, there was no allowance for the steps, gates or required documents of the project management process to be bypassed on this basis.
The final project manager, who took over the generation 3 RCE project in mid-December 2017 (shortly before commissioning of the generation 3 RCE), advised the ATSB that the initial project manager had advised them that the generation 3 RCE was the same as generation 2, with the addition of dynamic brake functionality, so the final project manager did not realise the extent of differences.
The final project manager further reported that:
very little documentation was passed on during project handover, with no documented RCE specifications, functions, or expectations
no-one within TasRail challenged the lack of documentation.
Project review and audits
Document BIC-PRO-100 (Project review and auditing) detailed the procedures to ensure compliance to TasRail’s change, risk, and project management processes. At the start of each financial year, the project management office’s senior project manager, chief operating officer, general manager asset management and general manager freight services would select 12 significant change projects for audit and review. The generation 3 RCE project was not selected as part of these reviews.
TasRail had additional audit requirements for projects funded under the state-federal partnership’s infrastructure improvement program. However, as an above-rail asset, the generation 3 RCE did not qualify as a project under the program, so these audits were not required. There was no audit of the generation 3 RCE project performed by TasRail’s project management office.
Risk management activities
Risk management requirements
An integral part of change management is identifying the hazards and assessing, controlling, monitoring and reviewing the risks associated with the change. TasRail’s method of risk management was contained in documents CGR-BCP-001 (Corporate governance and risk management) and CGR-PRO-100A (Risk management).
TasRail’s chief executive officer (CEO) delegated responsibilities for risk management. The general managers of freight services and asset management were responsible for ensuring the application of risk management principles to railway operations. The risk and compliance manager was responsible for the application of the risk management process, and assurance of compliance with the process (including risk control monitoring). Risk acceptance and ownership by TasRail management was based on a tiered risk ranking from T1 (low – project managers and supervisors) to T36 (extreme – CEO).
Assessment of risk from identified hazards was to be performed by ‘appropriately qualified staff’, using a qualitative TasRail risk assessment matrix. All identified risks, including environmental and health and safety risks, were to be contained within a risk register. The risk owner (the delegated accountable manager) was required to ensure the timely implementation of risk controls and arranging periodic reviews for critical and high risks (annually), and moderate and low risks (every 3 years). In addition, CGR-BCP-001 required a reassessment of risk and update to the risk register during a significant change, as triggered by the change management process (BIC-PRO-301).
Risk assessments for remotely-controlled train operations
Examination of TasRail records identified 3 safety risk assessments for the generation 2 RCE operations. These were for:
RCE commissioning, maintenance, and operations, conducted in December 2009 (prior to generation 2 RCE implementation) and revised in December 2011 (after implementation)
main-line driving between Devonport and Railton, conducted in January 2014
unloading and loading operations, conducted in June 2014.
The December 2011 revised risk assessment identified 39 safety hazards, including the following relevant hazards and associated risk controls:
maintenance or transit damage causing RCE ‘to fail safe or operate abnormally’, mitigated by safety checks, authorised maintainers and repairers, and proper shipping methods
failure of vigilance or tilt functions, mitigated by daily tests
loss of communications, with the relevant control stated as ‘loss of communication causes system to revert to a safe mode’
brake or throttle control failure on the transmitter, mitigated by:
This risk assessment did not explicitly define the meaning of ‘operate abnormally’ as it might relate to the potential for the RCE to fail to an unsafe state. Neither of the other 2 risk assessments identified the potential for a fail-to-unsafe[71] RCE event as a potential hazard.
The January 2014 risk assessment (RCE main-line operations) identified hazards related to driver unfamiliarity with RCE, level crossing collision and track warrant authority exceedance. For the authority exceedance hazard, the risk controls were driver qualification and a ‘proven remote control [RCE] fail safe system’. The method with which this fail-safe design was developed and how this had been proven was not explained further. Risks associated with the operational configuration of TasRail’s remotely-controlled trains (such as operating from the driver’s van while propelling at maximum track speed) were not assessed.
Among the hazards assessed in June 2014 were undesired train movement during loading and unloading, resulting in cement spill. The controls for the unloading hazard comprised driver competency and driver communication. For the loading hazard, the Railton loading interlock was the only nominated control.
There were no additional risk assessments conducted prior to the introduction of generation 3 RCE. During March–April 2018 (post-implementation of generation 3 RCE), TasRail completed a further risk assessment in relation to the risk of a collision resulting from failure of RCE on the cement train. There was no explanation about the type of failure assessed, or how the identified risk controls would mitigate the risk. Identified controls included (paraphrased):
driver competency
January 2014 (generation 2 RCE) risk assessment for main-line driving
new generation 3 RCE
driver’s van emergency brake valve.
Several additionally-identified risk controls relied on a radio link between the RCE transmitter and RCE receiver or continued functioning of the receiver. These were:
tilt function
vigilance function
communication failed mode
transmitter flat battery warning.
Only one of the risk controls, the driver’s van emergency brake, enabled operation independently of the RCE. However, it required the driver to be in the driver’s van to operate it, which was not addressed in the risk assessment.
The residual risk from the 2018 risk assessment was rated at the lowest end of medium, requiring monitoring. CGR‑PRO-100A classed this rating as follows:
Most controls are designed correctly and are in place and are reasonably effective.
Some additional action(s) are required to improve the effectiveness and reliability of controls to provide reasonable assurance that objectives will be achieved.
Design assurance processes
TasRail had 2 documents for the design and verification of rolling stock: RS-PRO-017 (Design procedure – rolling stock) and RS-PRO-016 (Design verification, validation, and acceptance procedure – rolling stock). Jointly, they provided process requirements such as:
responsibility involvement of relevant stakeholders
technical specification, which can include a design process, accreditation and engineering standards
a plan for how requirements and standards would be met
staged process reviews, including that ‘calculations and decisions for defined critical systems have been independently checked and verified’
risk assessment, including safety risks, and risk assessment workshops
review of identified failure modes (for reliability), including the suggested use of specialised techniques such as failure modes and effects analysis (FMEA)[72]
design and development planning
documentation requirements
configuration management
verification and validation
design endorsement, approval, and acceptance.
RS-PRO-016 stated:
Any change that results in a departure from the approved design shall be documented and may require further verification and validation prior to such acceptance being granted.
The process also stated:
Design acceptance can only be granted by a Certified [chartered] Engineer. This is required particularly for all designs and modifications that are classified as Safety Critical.
RS-PRO-017 and RS-PRO-016 were not referenced by, or otherwise linked to, other TasRail processes, such as those for change management and risk management.
These engineering processes stated the intent to meet the ‘design elements set out in AS [Australian Standard] 4292.1 and AS 4292.3’ (see Design assurance standards).
Regulatory oversight
Relevant legislation
In Tasmania, the Rail Safety National Law (Tasmania) Act 2012 adopted the Rail Safety National Law (South Australia) Act 2012 (RSNL) and the subordinate Rail Safety National Law National Regulations 2012. That is, the South Australian Act and the national regulations applied in Tasmania. Prior to the application of this legislation, rail safety in Tasmania was regulated by a state-based regulator (the Tasmanian Rail Safety Regulator).
General regulatory approach
The 2018 publication The ONRSR way[73] stated that ONRSR’s primary objectives were to encourage safe rail operations, ensure compliance with the RNSL and to promote and improve national rail safety, using a risk-based regulatory approach. This was done through a co-regulatory framework. Co-regulation, while not unique, was not common in the transport industry. The ONRSR way explained:
…The nature of the RSNL means that we are not a technical regulator.
The regulatory framework is co-regulatory in that the Australian governments do not directly prescribe the standards or rules by which railways need to operate. Rather, they set a performance requirement on railways to operate safely and provide operational flexibility to establish and implement standards, rules and methods of operation necessary to meet the safety performance requirement of their operations. The co-regulatory framework of the RSNL enables us to tailor our approach to each operator and their circumstances, while at the same time aiming to present a consistent regulatory approach to the rail industry so as not to surprise…
The co-regulatory framework is strongly founded on the distribution of responsibility for the management of risks to safety, which is expressed in the RSNL through the:
principle of shared responsibility for rail safety risks;
establishment of specific safety duties for rail transport operators, designers, manufacturers and suppliers to the rail industry, loaders and unloaders of freight and rail safety workers; and
establishment of the role and function of the Regulator [ONRSR]…
Co-regulation is not a partnership between ONRSR and the rail industry when it comes to the obligation to ensure, so far as is reasonably practicable, the safety of railway operations.
This responsibility clearly rests with those parties that are directly in a position of control and management of railway operations, which is principally the accredited or registered rail transport operators and their various suppliers and contractors.
The overall success of this regulatory framework to address and mitigate risks to safety is predicated on individual operators and the broader industry fulfilling their respective roles in engaging the appropriate expertise and competence towards collaboratively identifying and assessing risks and developing, applying and maintaining standards and processes to manage safe railway operations…
In terms of how this approach was intended to be applied, The ONRSR way stated:
We [ONRSR] have the dual, but complementary, roles of administrator of the RSNL accreditation regime and the regulator of a duty-based safety management regime (an educator, monitor, and enforcer). The nature of the RSNL means we are not an approver of equipment, services or processes.
In conducting these roles, we seek to engage with operators in a way that directly influences those with the ultimate responsibility for delivery of safe railway operations and environments. We take a predominantly facilitative approach to regulating safety, with our rail safety officers collectively acting as a safety conscience and compliance coach to our regulated parties, targeting education where necessary and giving opportunity for operators to address identified safety issues. However, where this is not effective with individual operators or more immediate, publicly accountable action is required, we will employ the range of enforcement options available to us to secure safe outcomes and compliance with the law.
The ONRSR Way stated that ONRSR’s role was to administer accreditation, monitor safety management performance, and ensure operators and duty holders comply with the requirements and safety management standard set by the law (the RSNL).
Rail transport operators (RTOs) were responsible for the establishment and implementation of the standards, rules and procedures for safe operation. They were also, along with contractors, suppliers, and manufacturers, accountable for ensuring (so far as is reasonably practicable) safe operations and activities. Rail industry standards groups (such as the Rail Industry Safety and Standards Board or RISSB) supported this through the development of ‘good practice' guidance or standards.
ONRSR’s monitoring, investigation and enforcement activities were undertaken through an array of regulatory activities, which were:
reviewing notifications of change, which may have impacted an RTO’s accreditation (see Notification of change)
enquiries, meetings, site visits, inspections, audits and investigations
advice, education, and non-conformance reports
review of operators’ safety performance reports or annual activity statements
notifiable occurrence response
issuance of statutory notices, such as improvement, prohibition and infringement notices
acceptance of enforceable voluntary undertakings
prosecutions
additions of conditions / restrictions, or suspension, or cancellation of RTO accreditation / registrations.
Notification of change requirements and guidance
General information
As outlined in regulation 9(1)(a) of the national regulations, accredited[74] RTOs were required to notify ONRSR of proposed decisions, proposed events or changes to their operations.
The regulation prescribed 12 items of railway operations to which this requirement applied (reproduced in Appendix C). No item numbers under regulation 9(1)(a) explicitly mentioned notification of changes to ONRSR relating to RCE. In response to questions by the ATSB, ONRSR advised that TasRail could have submitted a notification of change in relation to the generation 3 RCE based on one of the following notifiable items:
…Table Item 3 – change to a safety critical element of existing rolling stock, if the operator believed that the change to the remote control equipment (RCE) changed a safety critical function that it performed; or
Table Item 5 – change to safety standard for the design of rail infrastructure or rolling stock, if the operator believed the change represented a change to the expected or required safety performance (standards) of the equipment.
Timeframes for notifying ONRSR of a change were also nominated in the regulation, which were typically 28 days before implementation.
The ONRSR Notification of change to railway operations form required RTOs to:
provide a description of the change
advise change management procedures that were or would be followed
advise whether risk assessment of the change had been conducted
advise risk controls for the change that were, or would be, in place
advise whether consultation with affected parties was or would be conducted.
Regulator actions in response to a notification of change
The ONRSR Internal Framework – Risk based Regulation document described the regulator’s role of encouraging and promoting improvements to rail safety through ‘...an approach to regulation in which regulatory effort is commensurate with risk and scope for improvement’. The framework ranked ONRSR’s decisions in 3 tiers that determined their impacts to rail safety. Decisions related to notification of change and follow-ups were ranked as tier one decisions; that is, those that had the ‘greatest potential impact on rail safety…made to directly fulfil primary functions of ONRSR’.
The ONRSR internal notification of change process was detailed in the Notification of change procedure. The process required that the assigned branch accreditation manager would perform an initial review to determine if:
the change was within the RTO’s scope of accreditation, including conditions and restrictions (if not, change was not permitted and a variation of accreditation was required)
there was an immediate risk to safety posed by the change (if so, immediate regulatory compliance and enforcement action was invoked)
further information was required:
to assess the extent and impact of the change
to confirm that change management and risk management processes were being followed
in instances where there was significant change to the RTO’s SMS or risk profile.
Where further information was required, a rail safety officer (RSO) was then allocated the notification of change for follow-up and review. The assistance of ONRSR technical specialists could be sought to assist rail safety officers during this review process.
ONRSR did not approve or reject notifications of change. Rather, based on any further information received, previous RTO compliance history, the type of rail operation and SMS maturity, ONRSR could undertake further regulatory activities either prior or post the notified change taking effect. These included:
meetings
site visits
inspections
audits.
Guidance for the interpretation of the notification of change regulation
The ONRSR Way stated that, to provide a consistent regulatory approach, ONRSR aimed to (among other things):
...provide a single, common interpretation to the RSNL and our expectations on operators arising from this interpretation (made public through our published guidance material)…
…We have an important role of interpreting the RSNL, explaining this to regulated parties, and acting consistently with this interpretation…ONRSR develops comprehensive guidelines which clearly articulate our expectations in relation to complying with the RSNL. These guidelines provide key information and clarification to both the rail industry and public on legislative, regulatory and technical matters.
For the notification of change process, ONRSR provided guidance to RTOs and ONRSR staff within several documents:
ONRSR Notification of change policy, which provided direction on the requirements for a notification of change
ONRSR Regulatory response to the management of change policy, which defined ONRSR’s regulatory requirements for accredited RTOs planning change to their railway operations
ONRSR Notification of change guideline, which provided advice regarding requirements for the submission of a notification of change
ONRSR Notification of change fact sheet, which provided an overview of the process
The ONRSR Way, which provided an explanation of ‘how ONRSR intends to work … operationally and strategically’, including ONRSR’s functions and powers, and the application of the co-regulatory framework
ONRSR Notification of change procedure, which was for internal use only, and included:
guidance to ONRSR rail safety officers on key areas to be addressed in processing the notification of change that were not detailed in the rail safety legislation
responsibilities and accountabilities for tasks within the process
communication of information to the relevant parties (internal and external)
management of internal systems, including the records required to be captured throughout the process.
The ONRSR Notification of change policy stated that ‘routine operational and administrative changes that occur within railway operations’ should be managed by the operator’s management of change procedures and were not required to be notified to ONRSR. The available guidance for interpretation of the 12 items nominated in regulation 9 is discussed in the following subsections.
With the exception of The ONRSR Way and ONRSR Notification of change procedure, all of the above ONRSR documents contained a recommendation that an RTO ‘…contacts the ONRSR to discuss the change/s…’ prior to submission of the Notification of change to railway operations form if the RTO was unsure whether the change was within the scope of its accreditation or whether regulation 9(1)(a) reporting requirements applied.
ONRSR advised the ATSB that RTOs could contact either the head office (in Adelaide) or their local rail safety officer for notification of change clarifications. An ONRSR representative advised the ATSB that rail safety officers were expected to either: seek internal assistance (for example, the director of operations, legal counsel or the technical team) if unsure how to answer a clarification enquiry; or, alternatively, advise the RTO to submit the notification of change. ONRSR advised the ATSB that there was no specific internal guidance to assist rail safety officers when answering notification of change clarification requests.
Terms pertaining to the notification of change
With regard to item 3 in regulation 9, definitions for ‘safety critical element of rolling stock’ or ‘safety critical’ were not provided in the Rail Safety National Law (South Australia) or the Rail Safety National Law National Regulations 2012 (NSW).
The Notification of change guideline[75] included interpretations and examples for 3 of the 12 notifiable items in regulation 9(1)(a). These related to changes to rail infrastructure (item 4, one descriptive paragraph), network rules (item 8, one descriptive paragraph) and work scheduling practices related to fatigue risk management (item 10, 2 descriptive paragraphs). There were no interpretations and examples included for items 3 or 5.
The Notification of change procedure stated that its purpose was, in part, to provide rail safety officers with ‘…key areas to be addressed in processing the Notification of Change which are not detailed in the rail safety legislation’. However, this document did not provide a definition or examples of what constituted a ‘safety critical element of existing rolling stock’, the interpretations of the terms ‘safety critical’ or ‘change’.
Interpretation of ‘element of ... rolling stock’
The RSNL and Regulations defined ‘rolling stock’ as:
…a vehicle that operates on or uses a railway, and includes a locomotive, carriage, rail car, rail motor, light rail vehicle, train, tram, light inspection vehicle, self propelled infrastructure maintenance vehicle, trolley, wagon or monorail vehicle…
RCE was not explicitly defined in the legislation or the related guidance (as an element of rolling stock or otherwise). ONRSR advised the ATSB that, in relation to generation 3 RCE, it had ‘…the primary purpose of enabling the driver to control the movement and operation of the locomotive, [therefore it] is considered an element of the rolling stock (locomotive)’.
Interpretation of ‘safety critical’
The Notification of change fact sheet included the following definition:[77]
Safety critical elements: are identified in the [RTO’s] safety management system and include any component part of equipment, plant or system whose failure could substantially contribute to a major accident.
Additionally, ONRSR advised the ATSB that:
Safety critical, is not defined in the RSNL and scope is given for an operator [RTO] to interpret this in the context of their own railway operations. It is typically taken to mean elements of rolling stock that are critical to the safe operation or movement of rolling stock, particularly in regard to preventing unsafe/uncontrolled operation or are critical in being able to respond to mitigate or prevent risks to safety of persons, includes but not exclusively, braking systems, coupling, communications, safety recording devices, automated train protection systems etc.
In addition, prior to commencement of the RNSL and national regulations, each state and territory had its own rail safety legislation, including definitions. The Rail Safety Regulations 2010 (Tasmania), which required a notification of change to the then-regulator for Tasmania for ‘a change to a safety-critical element of existing rolling stock’, did not define ‘safety critical’.In contrast, Schedule 5 of Queensland’s repealed Transport (Rail Safety) Regulation 2010 included the following definition:
safety critical element, of rolling stock, means any part of the rolling stock that could cause, or affect the impact of, an accident or incident if the part failed to operate properly.
Interpretation of ‘change’
ONRSR advised the ATSB that, although guidance on interpretation of the extent of change notifiable for regulation 9(1)(a) was not provided to RTOs, internally it would consider a change notifiable under item 3 if:
…the change involved a change to how a safety critical element of rolling stock is likely to perform or is operated. A like-for-like (replacement/repair) would not be considered a change but a minor/major redesign for the element to behave differently or require ‘re-training’ for use would.
Other definitions of ‘safety critical’
Appendix E of AS 4292.3‑2006 (Railway safety management, part 3: rolling stock), effectively superseded in 2017, provided a recommended list of 25 items that should be considered ‘safety critical items’ on rolling stock.[78] Although RCE was not mentioned on this list, ‘braking systems’ and ‘event recording equipment’ were included. The information provided in the standard was not reproduced among the RISSB products that superseded AS 4292.
Australian standard AS 7501:2013 (Rolling stock compliance certification) did not define ‘safety critical.’ A related standard, AS 7702:2014 (Rail equipment type approval), defined it as ‘directly influencing safety (when applied to equipment or systems).’ Notably, this standard specifically excluded rolling stock in its scope, but allowed for application of its principles to rolling stock, except where there was conflict with AS 7501.[79]
AS 7702 also required the supplier to provide documentary evidence of safety and specified that products typically require type approval if (in part):
the product is used in a safety critical function where its failure can directly affect the safety integrity of the railway
the product is used in a function where its failure will affect the railway.
Generation 3 remote control equipment notification of change
ADE advised the ATSB that, although it had interactions with the Tasmanian Rail Safety Regulator during development of generation 1 RCE, it did not interact with ONRSR for generation 3 RCE.
TasRail defined ‘safety critical’ in its documents RS-PRO-016 (Design verification, validation and acceptance procedure – rolling stock) and RS-PRO-017 (Design procedure – rolling stock) as ‘any element whose failure or malfunction may result in death or serious injury to people, or loss or severe damage to equipment or environmental harm.’
On 19 December 2017, in response to a request by the final project manager, TasRail’s risk and compliance manager sought clarification from ONRSR to confirm whether a formal notification of change was required. The manager advised the ATSB that they had a phone conversation with an ONRSR rail safety officer, during which several items were discussed, including the generation 3 RCE project. The TasRail risk and compliance manager recalled that they advised the rail safety officer the RCE was like-for-like with the addition of a dynamic brake.
The ONRSR rail safety officer made a note of this conversation in their official notebook for recording their activities and interactions with RTOs:
Replacement / upgrade of Remote Control System on Cement Train.
The locomotive remote control unit will be replaced shortly. [The TasRail risk and compliance manager] advised this is a like for like replacement of the existing unit which has been in operation for some time and due for routine replacement. I confirmed to [the TasRail risk and compliance manager] that this is not a notifiable change.
There was no mention in this note that the manager had advised the rail safety officer of the addition of dynamic brake to generation 3 RCE.
Consistent with this conversation, TasRail did not submit a formal notification of change in relation to the generation 3 RCE project.
Previous TasRail activities regarding the notification of change
A review of the notification of change submissions received by ONRSR determined that 15 were submitted by TasRail between May 2015 (commencement of the generation 3 RCE project) and the day of the accident. Of these, 6 were related to introduction, re-introduction or changes to rolling stock. Seven required requests for further information by ONRSR, including 3 that had been submitted incomplete.
ONRSR advised the ATSB that it did not receive a notification of change from TasRail relating to the conversion of the DQ and 2050 class locomotives and generation 2 RCE from a 3-pipe to 4‑pipe braking configuration in 2014.
Occurrence notification
TasRail was required to report ‘Category A’ notifiable occurrences immediately to the ATSB,[80] with a follow-up written report within 72 hours to ONRSR. For ‘Category B’ notifiable occurrences, TasRail was required to provide ONRSR with a written report within 72 hours of the occurrence.[81]
Depending on the type of failure, failure events of generation 3 RCE could have been considered notifiable occurrences under the following legislative subparagraph:
57(1)(b)(xvii) the detection of an irregularity in any rolling stock that could affect the safety of railway operations [Category B].
TasRail’s risk and compliance manager was responsible for advising the ATSB and / or ONRSR of Category A and B notifiable occurrences. These were based on risk wizard entries, which, in the case of RCE failures, were created by the NCO receiving the RCE driver’s report (see Fault reporting process).
A review of both the ATSB and ONRSR notification and occurrence databases found no reports of notifiable occurrences received from TasRail in relation to generation 3 RCE, apart from the accident on 21 September 2018. The risk and compliance manager advised that instances of RCE failing-to-safe were likely not recognised by NCOs as being of a safety concern and therefore not placed in risk wizard.
Requirements and guidance related to system safety
Safety management systems
The RSNL regulations required operators to have safety management systems (SMS) with 30 elements that included:
a safety policy
safety culture
governance, compliance and internal control arrangements
individual accountabilities
safety performance measures and audit arrangements
change management and risk management procedures
engineering standards and procedures, and operational systems, safety standards and procedures, including for engineering design of rolling stock.
ONRSR also provided guidance on SMS implementation.
Major projects guidance and expectations
In 2014, ONRSR issued ONRSR Guideline – Major Projects to ‘provide guidance to the industry as to how projects can safely manage major change.’ The guideline stated:
In the context of this guideline, a major project is considered to be an entity, or entities, that are managing the delivery of a significant change to railway infrastructure or rolling stock.
There was no definition for what constituted a ‘significant change’ in this context, although the guideline separately stated that major projects involve multi-disciplinary activity, complex contractual structures, and intricate organisation structures.
The guideline was not intended to apply to complex, safety-critical projects that did not include these types of activities and structures, such as the development of the generation 3 RCE. However, it referred to methods, standards and other guidance on systems engineering and safety assurance principles that could be beneficial for complex, safety-critical projects other than major projects. These included:
safety assurance planning
independent safety assessment
documented safety limits (that is, the limits of acceptable risk)
human factors integration, including human-system interface assessment and risk-based training needs assessment
‘good practice system engineering principles’ such as safety requirements management, ‘V‑model’ principles (see Appendix B), use of relevant safety standards such as IEC 61508 and EN 50126/8/9, and management of project-specific risks.
Design assurance standards
There was no regulatory framework for national or state-wide type approvals for rolling stock. Rather, there could be a form of type approval process within the SMS (as TasRail had). Similarly, there were no prescribed regulatory requirements for RTOs to follow systems engineering or system safety design processes for rolling stock or infrastructure other than those nominated by the RTO itself.
There were voluntary Australian Standards (AS) relevant to rail safety, including:
AS 4292 Railway safety management (withdrawn—see below)
AS 7501 Rolling stock compliance certification
AS 7702 Rail equipment type approval.
AS 4292 (Railway safety management) was ‘prepared primarily with a view to achieving uniformity in the management of railway safety both as a general principle and with specific reference to the accreditation of railway industry participants.’ It comprised 6 parts (AS 4292.1 through AS 4292.5, and AS 4292.7).[82]
AS 4292.1 contained information about safety management systems, risk management, and related processes, including engineering design control processes. It included the requirement to ‘implement and maintain a documented set of engineering and operational systems safety standards’ to cover all relevant aspects of rolling stock and other systems and infrastructure. With respect to engineering design, it required:
(a) Identification of the responsibility for each design or development activity.
(b) Safety risk review at both the design input and design output stages taking into account reliability and maintainability.
(c) Assignment of design verification and validation functions.
(d) Control of design changes in accordance with Items (a) and (b).
AS 4292.3 provided information about establishing processes for the design of rolling stock. It included:
a recommendation to identify hazards that might affect the ‘integrity’ of rolling stock and trains
a requirement to establish standards and procedures for the selection and design of rolling stock, including braking systems, electrical equipment, and ‘operation of rolling stock, safety elements’
a requirement to establish standards and procedures for the verification and validation of all stages of the design of rolling stock, including conformance to safety requirements and standards, with an ‘appropriate level of independence from the design process’ depending on the risk of process error and the overall level of safety risk
a recommendation to have an inspection and test plan, including ‘verification that the system conforms to the design and operating requirements of the client and the operating parameters of the railway’ and ‘validation that the installed system conforms to the required safety standards and client requirements’
a recommendation to include ‘commissioning tests’ for ‘operation of rolling stock safety elements’, which included driver’s controls and warning devices
a recommendation for monitoring to identify ‘safety critical faults’
a recommendation for consideration of the effects of modifications to rolling stock on the railway system as a whole, and the need to follow design processes for the modification
recommended lists of items to be considered when developing performance standards during the design stage, which included ‘safety critical items’ such as braking systems.
In 2017, Standards Australia announced that the 6 parts of AS 4292 were to be withdrawn. RISSB advised in a technical note to industry that most components from AS 4292 had been absorbed into other standards, codes of practice and guidelines.[83] AS 4292 was withdrawn in 2021.
AS 7501(Rolling stock compliance certification), first issued in 2013, included process requirements for the design or modification of rolling stock. These requirements included independent verification by a person with (among other attributes) demonstrated experience and knowledge of the requirements of AS 4292 and experience in assessing rolling stock against standards. There was no requirement or recommendation to produce a safety case or conduct specialised hazard analysis techniques such as FMEA.
The comparable voluntary standard primarily intended for use with rail infrastructure (AS 7702 Rail equipment type approval), first issued in 2014, had the following requirements:
The supplier shall provide documentary evidence of satisfying the RTO’s safety requirements.
In relation to the above clause the supplier should provide details and reports for any safety analysis such as Safety Integrity Level demonstration (including documentation on which any report is based) either carried out by the Supplier, or independent reports…
The Supplier should provide details of:
failure modes, degraded modes (effect on operation of partial failure), emergency operation;
the effect on railway operations of degraded modes of failure;
emergency operations;
fire performance: flammability and smoke;
vandal resistance.
AS 7702 additionally stated:
Safety may, for example, be demonstrated by Independent Safety Assessor Report, Safety Case document or FMECA[84] analysis with risk assessments being undertaken in accordance with AS/NZS ISO 31000 [Risk Management].
In other words, this voluntary standard for rail infrastructure identified the use of a safety case or analysis to support the evaluation of system safety, but this guidance was not present in the equivalent standard for rolling stock.
Event recorders
Event recording used on TasRail cement trains
Three devices with recording capability were fitted to the TR class locomotive:
a forward-facing, closed-circuit television
a dedicated event recorder
the system for automated locomotive computer control (SAL).
Parameters recorded by the event recorder and the SAL differed to some extent, and both recorded parameters that were not replicated on the other. Although not its primary function, the SAL recorded parameters relating to locomotive control.
A data recording device was also fitted to the driver’s van, which was only capable of recording driver’s van headlight and horn operation. With no link to the locomotive or RCE, the usefulness of any data recovered would have been minimal.
A data logger was not included in the generation 3 RCE. TasRail enquired about the addition of data recording to the generation 3 RCE on 2 further occasions in May and June 2018, both times coinciding with RCE operational events. This request progressed as far as ADE providing a quote for the inclusion of a recording capability in June 2018 at an additional cost of about 4% of the total cost of the 2 RCE sets; TasRail had not progressed this further by the time of the 21 September 2018 runaway.
Requirement to use an event recorder
There was no explicit requirement in the RSNL or the national regulations for a train or locomotive to be fitted with an event recorder. ONRSR personnel advised that modern locomotives were generally fitted with an event recorder.
TasRail’s accreditation, as RTO, which was initially issued by the Tasmanian Rail Safety Regulator, was transitioned across to ONRSR in January 2013.[85] The transitioned notice of accreditation included conditions and restrictions. One condition related to event recorders, which stated:[86]
All leading locomotives or other vehicles used as the leading vehicle of a train shall be fitted with an operating event recorder.
The condition was not restricted to any type of train, and therefore applied to TasRail’s remotely-controlled train operations as well as its other train operations.
As previously stated, when operating the cement service, the locomotive was the leading (front) vehicle when travelling to Devonport and the driver’s van was the leading (front) vehicle when travelling to Railton. However, the RCE receiver was providing the driver’s interface and command outputs to the locomotive at all times. Therefore, although TasRail did not explicitly define a ‘leading vehicle’ in the cement train consist, it was reasonable to consider that the RCE was fulfilling this function because it assumed the controlling role of a lead locomotive over airbrake, throttle, direction, and other safety-related controls such as the vigilance function.
ONRSR described in The ONRSR Way that one of the purposes for accreditation conditions and restrictions was:
…to secure a specific method of operation or use of technology that the operator [RTO] has committed to in order to achieve accreditation (meaning that the operator [RTO] will need to apply for a variation to accreditation to change this).
ONRSR advised the ATSB that the ‘condition on requirement for event recorder’ was a legacy condition transitioned from the previous state regulator upon changeover to the Rail Safety National Law. However, The ONRSR Way indicated that ONRSR did have the discretion to remove conditions unilaterally:
…we will monitor the effectiveness and relevance of what [conditions] we have imposed. We will act to remove these, without initiation by the operator, should we decide that the condition or restriction is no longer needed for our purposes.
An ONRSR representative stated that it was not common for a condition related to event recorders to appear on an RTO’s accreditation notice. Although they were not aware why this legacy condition was on TasRail’s notice of accreditation, they believed it would have been for a safety reason.
The Tasmanian Department of State Growth indicated that it did not retain a copy of the previous notification of accreditation permitting the use of remote control locomotives or records indicating why the requirement for event recorders was originally applied.
In relation to the event recorder condition on TasRail’s notice of accreditation, ONRSR advised the ATSB:
Whilst the condition calls specifically for the event recorder being fitted to the leading vehicle, ONRSR considers an event recorder fitted anywhere in the consist, that is recording the key parameters, as meeting the intent of the condition.
There is no requirement to log control inputs of remote control equipment.
ONRSR did not provide further definition or explanation of what were considered ‘key parameters’.
TasRail were requested to provide safety management system (SMS) documents relating to requirements for event recorders on its trains. In response, it provided a procedure that applied to ‘the collection and retention of Data Logger data’. The procedure did not include any requirements or references stating what parameters had to be recorded. Therefore, there was no evidence provided to indicate what parameters TasRail required to be captured on its event recorders in order to meet its condition of accreditation.
Following an accident involving an inter-urban passenger train at Glenbrook, New South Wales, in December 1999, which resulted in 7 fatalities, a Special Commission of Inquiry noted that there was no data logger (event recorder) fitted to the train, and that this hampered the effectiveness of the investigation. In relation to this aspect, the inquiry final report made the following recommendation:
All trains be fitted with data loggers to enable, among other things, train driver performance to be monitored.
Following an accident involving a suburban passenger train at Waterfall, New South Wales, in January 2003, which resulted in 7 fatalities, a Special Commission of Inquiry noted that there was a data logger fitted to the train but it was not operational at the time of the accident. The commission’s report found that the absence of recorded data hindered the establishment of important facts. The commission final report stated (McInerney 2005):
The data loggers, and the data they provide, are crucial to the monitoring and improvement of the reliability of rolling stock, because of the data obtained relating to the performance and maintenance of the rolling stock…They are an essential piece of equipment for the determining of the causes of incidents and accidents…
Data loggers are such an important piece of equipment that, in my opinion, it should be mandatory for all trains to be fitted with them and rail organisations should not be accredited to operate on the New South Wales rail network unless each train is fitted with an operational data logger. This requirement should be a condition of accreditation for any organisation seeking to operate on the New South Wales rail network.
AS 4292.3:2006 (Railway safety management, part 3: rolling stock), effectively superseded in 2017 as described in Design assurance standards, listed ‘event recording equipment’ among a list of ‘safety critical’ items ‘to be considered when developing performance standards … for new or modified vehicles.’
AS 7527:2015 (Rolling stock event recorders) ‘describes the requirements for event recorders installed in locomotive… rolling stock vehicles’. Although use of the standard was not mandatory, it gave RTOs guidance about which parameters an event recorder should monitor in order ‘to ensure that event recorders fitted to rolling stock capture a minimum set of appropriate data for the use of rollingstock operators, rail infrastructure managers, investigators and maintainers for forensic investigation of rail incidents’.
The standard specified what information was required to be recorded in order to achieve compliance with the standard. It separated the information into either ‘mandatory’ or ‘recommended’ requirements.[87] The standard also indirectly contained requirements for some parameters to be displayed to the driver.
Appendix D provides a review of parameters that were identified as relevant to the 21 September 2018 runaway accident, the requirements detailed in AS 7527, and comment on their application to RCE operations. In summary, some mandatory and recommended data relevant to the RCE (such as driver inputs and the vigilance function) were not recorded when using the RCE. In some cases, the locomotive response to driver input was recorded but this did not meet the intent of the standard (for example, train direction was recorded at the locomotive, but direction as selected by the driver on the RCE transmitter was not recorded).
Other specific requirements in AS 7527 for a ‘driverless vehicle or remote-operated vehicle’ were not recorded on TasRail trains under RCE control, including:
all changes to the operational status of the vehicle (mandatory)
all control messages received and transmitted by the vehicle (mandatory)
all status messages received and transmitted by the vehicle (recommended).
Effect of non-recording of remote control equipment data
Although the TasRail locomotives did have data recording capability, they did not record the functioning of the generation 3 RCE (that is, parameters relating to the RCE modes, states, inputs, and outputs). This hindered the TasRail and ADE investigations into the Caroline Creek event and other potential precursor events. One of the TasRail representatives who had worked with ADE to address identified faults with the generation 3 RCE told the ATSB that recording capability would have assisted in both fault finding and rectification.
TasRail representatives asked ADE in February 2018 if there was a data capture port fitted to generation 3 RCE after multiple faults were encountered, and in May 2018 further indicated that data logging capability would be beneficial and help determine and/or understand the root causes of any issues.
ADE did not directly address the addition (or the absence) of data recording on the RCE in correspondence, but indicated that it was experiencing difficulties replicating faults reported on the cement train. For example, ADE suggested to TasRail that, upon drivers encountering unrecoverable faults, drivers should record video of what the RCE was reporting and doing when trying to recover to assist in fault condition identification.
The absence of recorded RCE data significantly hindered investigations into the runaway accident. There were insufficient parameters recorded to identify or replicate the failure state that the RCE had encountered, or to determine the states, modes, inputs and outputs of the RCE with a high level of confidence.
TasRail’s internal investigation report noted that the absence of recorded RCE data meant that investigators could not determine why communication was not restored. Similarly, ONRSR’s enquiries into the accident noted that the absence of recorded data prevented a full examination of driver actions and RCE receiver responses and recommended that the recording of RCE data should be considered for any future remotely-controlled train operations.
Safety analysis
Introduction
On 21 September 2018, TasRail freight train no. 604 rolled away from a cement loading facility at Railton, Tasmania. There was no train crew on board the train at the time. Travelling predominantly on down gradients toward Devonport, the train’s speed was uncontrolled and it exceeded the maximum track speed in place to ensure safe travel through curves, level crossings and points. The train travelled for about 20 km before being routed into a dead-end siding in Devonport, where it subsequently derailed, resulting in damage to the locomotive, 7 cement wagons and some surrounding infrastructure. Two pedestrians within the public area received minor injuries from fence debris.
Given the nature of the occurrence, the extent of injury and damage could have been much worse. A train runaway can cause injury or loss of life, substantial damage to rolling stock and infrastructure, and disrupt rail operations for an extended period. Accordingly, rolling stock operators and rail infrastructure managers need to implement sufficient preventative and mitigating controls to manage this hazard.
In this case, TasRail’s cement train service was operated using remote control equipment (RCE). It commenced these RCE operations in August 1999, with RCE designed and built by Air Digital Engineering (ADE). Subsequently, this was upgraded in about August 2010 to ADE’s generation 2 RCE, with TasRail commissioning ADE to manufacture the replacement generation 3 RCE in May 2017. Final delivery and commissioning of generation 3 RCE, the equipment used on the runaway train, occurred in February 2018.
It is almost certain that the cement train’s TR class locomotive and the locomotive and wagon airbrake systems were operating normally under the control of the RCE on the day of the accident. In addition, the driver was trained and considered competent in RCE operation, and their actions did not contribute to the accident in any reasonably foreseeable way.
Accordingly, this analysis will focus on the performance of the RCE and the controls in place to assure its performance and recover from any failure. More specifically, the analysis examines:
the performance of the generation 3 RCE during the runaway sequence
the risk controls in place to prevent an unsecure train from entering the main line
the processes for responding to emergencies such as a runaway train on the main line
design and integration problems associated with the generation 3 RCE
the system safety assurance process used by the RCE manufacturer
the rolling stock operator’s processes for acquiring RCE and managing its operations
the notification of change to the regulator and guidance for making such notifications
the rolling stock operator’s change management process
the rolling stock operator’s process for fault tracking and analysis
the recording of RCE functions and the associated requirements for recording
the guidance available for applying system safety assurance processes in the Australian rail industry.
Remote control equipment in unsafe state
A few minutes prior to the runaway, the driver was nearing completion of loading operations at Railton. During positioning of the last 2 wagons for loading, using the RCE transmitter, the driver misjudged the stop, which required the train to be reversed back into position.
Recorded data from the locomotive showed that, as the train stopped, the direction controller state quickly changed from forward to reverse (without a recorded pause in neutral). Following this:
there was an immediate venting of the brake pipe, equivalent to an emergency brake application, followed by a recharge of the brake pipe
at about this time, the RCE ceased to respond to driver commands
there were a further 2 subsequent emergency brake applications and releases.
Although there was no inherent safety concern with overshooting the stopping mark or reversing the direction, the ATSB later observed during multiple tests that a fast direction change (using the RCE transmitter) resulted in an unintended and momentary activation of the dual direction fault interlock and then emergency vent and recharge of the brake pipe.
The dual direction fault interlock function was designed to activate when forward and reverse direction outputs were detected as being simultaneously selected. However, an unintended consequence of this interlock function was an emergency brake application. The rapid decrease in brake pipe pressure it created resulted in activation of the locomotive’s VX vent valves and, in turn, emergency magnet valve (EMV).
Recorded data from train no. 604 showed similar brake behaviour coincident with the change from forward to reverse, indicating that the driver rapidly changing the direction on the RCE transmitter’s direction controller resulted in an unintended momentary emergency brake application due to activation of the dual direction fault interlock on the RCE receiver.
The dual direction fault interlock was not a feature that existed on the train’s TR class locomotive. Further, the length of travel between forward and reverse positions on the locomotive’s direction controller (during non-remotely-controlled train operations) limited the speed with which a change of direction could occur. However, on the RCE transmitter, the small size and ability of the direction controller switch to bypass the gated neutral position allowed almost simultaneous changes of direction.
The dual direction fault interlock function was not described in the RCE operation manual, and it was not assessed during driver training for the generation 3 RCE. A dual direction fault interlock event could also be cleared without intervention by a driver. Therefore, it is likely that the driver of train no. 604 was unaware that a quick change of direction could result in a momentary uncommanded emergency brake application, as occurred during the period prior to the runaway. Certainly, the driver could not have reasonably expected that a fast direction change would result in the RCE becoming unresponsive or lead to a runaway.
Although ATSB testing was able to reproduce the emergency vent and recharge of the brake pipe, testing of fast direction changes did not reproduce the unresponsive state or the following additional events that occurred immediately prior to the runaway:
the RCE ceasing to respond to driver commands
2 subsequent emergency brake applications
the final release of all train brakes.
Even though the nature of the persistent unsafe state could not be fully understood, the investigation could still determine factors that increased the likelihood of the state occurring, and decreased the ability of the RCE or other systems to prevent or manage the effects of the state. These factors, and others relating to TasRail’s remotely-controlled train operations using the ADE RCE, are discussed throughout later sections of this report.
Given the timing of the fast direction change before the immediate uncommanded emergency brake application (and subsequent events), and the absence of any other apparent trigger mechanism, it is likely (but not certain) that the fast direction change was the initiating event of the runaway. By comparison, the Caroline Creek event in May 2018 similarly involved an emergency brake application, but it was probably not initiated by a fast direction change (as the train was en route at the time).
The driver advised the ATSB that, after changing direction from forward to reverse on the RCE transmitter, they applied traction power. On noticing that the train did not respond, the driver observed the message ‘Connecting 9863 v1.4’ displayed on the RCE transmitter display screen. This message indicated that the transmitter did not have a communication connection to the receiver.
Accordingly, the driver initially treated this as a communication failed mode event (that is, where the receiver had detected communications between the transmitter and receiver had been interrupted for more than 4 seconds). As such, the driver attempted to reset the fault, by configuring the transmitter with full independent brake, automatic brakes applied, throttle to idle, and direction in neutral.
After observing that this was unsuccessful, the driver reported maintaining the RCE transmitter in the reset configuration while they attempted the control fault mode reset (that is, turning the transmitter off and on via the transmitter’s key switch). The driver advised that once the train commenced to roll away, they also applied a full-service automatic brake application, followed by tilting the transmitter (which should have activated the tilt function and applied the emergency brake) and removing the transmitter’s battery (which should have invoked a communication failed mode brake application).
However, the data recorded on the locomotive did not show locomotive responses consistent with the throttle application, the configuration of the controller required to recover from a penalty mode, or the actions the driver took to attempt to stop the moving train (key switch reset, tilt function or communication failed mode). Instead, the 2 emergency brake applications and releases were observed, followed by all train brakes being released. The configuration of the locomotive during these emergency brake applications, with no independent brake application apparent and the direction remaining in reverse, was not consistent with the RCE’s emergency mode or any other documented RCE fault condition.
The driver also thought they operated the transmitter’s emergency stop switch, although exactly when in the sequence could not be determined. However, this switch relied on a radio link to function, and based on the driver’s observations, the radio link was lost at the beginning of the non-responsive period. Therefore, it is highly unlikely that an emergency stop command would have had any effect. The other actions that the driver recalled performing (attempting to invoke a brake application by using a key switch reset and tilting the remote, and removing the battery) were appropriate and should have commanded the train to apply brakes and stop.
When the RCE software intentionally commanded an emergency brake application, the RCE receiver would enter its defined emergency stop mode. In addition to creating an emergency vent of the brake pipe, it was designed to command full independent brake and to set the direction to neutral. However, responses to these commands were not observed to occur during the runaway sequence. Therefore, the emergency brake applications that did occur during the sequence were very likely not a response to the software-commanded emergency stop mode.
In summary, the driver’s fast direction change from forward to reverse almost certainly (and unintentionally) resulted in a spurious fault condition associated with the dual direction fault interlock, which applied emergency brake. Through an undetermined mechanism, this likely initiated further anomalous RCE behaviour: the RCE ceased to respond to the driver’s commands, and 3 emergency brake applications and releases were recorded that were not consistent with previously identified fault state(s). A short time later, while still unresponsive to driver commands and with the train’s brakes still released, the train started to roll away.
Absence of runaway protection at Railton
The siding at Railton used for cement train loading was located on a downhill grade towards the main line in the direction of Devonport. The gradient posed a risk of a rolling stock runaway in the event of loss of braking effort.
An added risk unique to the Railton siding was the loading process, enabled by the use of RCE, where the driver was away from the operating train for a substantial period. This meant that multiple conventional means for immediate intervention by a driver in the case of an uncommanded movement were not available. For example, if the driver was in the locomotive cabin or the driver’s van, they could apply the locomotive brakes using the cab handle unit or the in-cabin emergency brake.
Additionally, TasRail’s advanced network train control system (ANCS) was deactivated on trains that were within yard limits. This included the cement train when it was within the limits of the Railton cement siding. Consequently, the ANCS system would not detect a runaway train leaving a yard and entering the main line. Therefore, no authority exceedance alarm would occur, and the runaway train would not be visible to the NCO. This limitation of the train control system increased the reliance on other systems existing that could prevent unauthorised movements from exiting yards onto the main line, and alerting the NCO if a runaway occurred.
TasRail conducted a risk assessment for the hazard of runaway rolling stock during March–April 2018. An action arising from the risk assessment was for TasRail to assess its network for the addition of catch points or derailers, the purpose of which was to divert and derail uncontrolled rolling stock movements from a siding away from the main line. At the time of the accident, the cement-loading siding at Railton had not been assessed and there were no devices in place to prevent or protect against unauthorised movement (such as derailers or catch points), the absence of which allowed the cement train to exit the yard onto the main line.
Remote control equipment response outside radio communication range
Once the RCE transmitter and the RCE receiver were linked (necessary for train operation), the RCE was designed to enter the communication failed mode if communications between the 2 components were interrupted for over 4 seconds. These interruptions could have resulted from corrupted messages, loss of radio communications or otherwise unlinking (for example, switching off the transmitter or pulling out the battery). During this mode, the receiver was supposed to stop the train by removing traction power, reducing the brake pipe pressure to 400 kPa and, after 10 seconds, changing the direction to neutral.
As previously discussed, the driver described the transmitter and receiver as being unlinked at the time the RCE became unresponsive to their commands. In addition, the driver reported they subsequently turned off the transmitter key switch and removed the battery, both of which should have interrupted communication and invoked a communication failed mode response had they still been linked. Even if the driver had not performed these actions, once the receiver was out of radio range of the transmitter (after the train began to roll away), it should have experienced a loss of communication and entered the communication failed mode, which should have applied the brakes.
Instead, a review of the data recorded on the locomotive showed that, after the second emergency brake application, the brake pipe pressure was maintained at the released level and there was no brake cylinder pressure indicating that the brakes on the wagons and locomotive were released. The recorded data also indicated that there was no brake application during this period.
Further, at the point of the derailment and rupture of the brake pipe in Devonport (over 20 km from Railton), the recorded data indicated that it was more likely than not that the RCE was supplying air from main reservoir no. 1 in an attempt to recharge the brake pipe to the desired level, which continued until the locomotive was shut down by emergency services. That is, the recorded data showed no evidence of the RCE receiver commanding a communication failed mode and applying the train’s automatic brake when out of radio range of the transmitter.
The overspeed protection on the TR class locomotive was set to apply brakes if the measured speed reached 88 km/h. In this case, the train’s maximum recorded speed during the runaway was 87.5 km/h. However, even if the train had reached 88 km/h, the overspeed protection would not have applied because the function was suppressed when the locomotive was configured in ‘trail cut-out’ mode for use with the RCE (see Suppressed or absent safety functions).
In summary, after the train rolled away from Railton, the RCE receiver did not apply the train’s brakes as designed when outside radio communication range. The train’s brakes remained in a released state until the train reached the end of the line and derailed at Devonport.
Emergency response actions and procedures
Actions by key personnel
As previously noted, after the RCE became unresponsive to driver commands, the driver attempted to reset it using the transmitter. Shortly after, the train began to roll away. Although the driver made further attempts to stop the train using the transmitter, these were unsuccessful. At this point the driver called the network control officer (NCO) to report the runaway train.
Although the NCO discussed with the driver where the train would likely stop, background conversations of other TasRail staff in attendance determined that the best course of action was to route the runaway train away from the main line to derail in Devonport Yard, and arrangements were made to configure the points at the entrance to the yard to achieve this. On conclusion of the phone call with the driver, and in accordance with the TasRail emergency response procedure, the NCO called emergency services to report the runaway train.
Over the next 16 minutes the NCO (and later the network access manager) liaised with emergency services to assist in the response. They were assisted by the rolling stock asset manager who monitored the train in real time, allowing emergency services to be advised of train location and speed.
This critical information allowed a coordinated and effective response by Tasmania Police, who deployed to close level crossings in front of the train. Although emergency services were quickly advised, by the time the Tasmania Police response was initiated the train had passed many of the level crossings between Railton and Spreyton township. Police also alerted members of the public to the danger through activation of lights and sirens on all attending police vehicles. When achieved, these actions counteracted the increased risk of level crossing collisions from the reduced warning time at the active crossings (due to higher train speed) and lack of train headlight for visibility and horn for audible warnings at all crossings.
Although the train was seriously damaged during the derailment accident, injuries were limited to 2 pedestrians who received minor injuries. All persons involved in managing the runaway train on the day, including TasRail and emergency services, acted to effectively minimise the consequences of the runaway and derailment.
Processes for responding to emergencies
Although the emergency response was successful on this occasion, this was not assured. TasRail’s emergency response procedures were generic in nature, requiring an NCO to immediately protect an area and notify emergency services. However, the procedures provided little other guidance or assistance to an NCO dealing with specific types of incidents that may require a time-critical tailored response, such as a runaway or track warrant authority exceedance. For example, the effective management of the Railton runaway involved sound decisions (routing the runaway train away from the main line and coordinating with emergency services to close level crossings in front of the train) that were not included in the emergency procedures but could have been considered in advance.
It is recognised that emergencies can take many forms and, as such, decisions and procedures require some level of flexibility. However, emergency responses involve high workload, distractions, interruptions, time pressures and stress. This can result in an environment where omissions and errors can easily occur. The Rail Industry Safety and Standards Board (RISSB) guideline document Rail emergency management planning stated:
Maintaining ‘quick references’ is recommended to support response and recovery. In the initial onset/identification of the emergency, Procedures or Action Cards and Checklists should detail immediate and important actions to help workers at all levels perform effectively.
The absence of a formal quick-reference procedure or checklist to deal with specific types of incidents meant there was a significant risk that NCOs, when dealing with events requiring time-critical decisions and response, would make errors and omissions.
Two types of checklists would be useful in this regard. The first is ‘read-do’, where the checklist item is read and the action performed, which is particularly useful when performing an infrequent task (Dismukes and Berman 2010). The second is ‘flow-then-check’, where several actions are performed from memory, then verified as completed through a checklist. However, as useful as these are, both types of checklist require firm underlying procedures to be most effective.
Another aspect worth noting is that the runaway occurred during normal business hours when extra staff were present who contributed to share the NCO’s workload in dealing with the emergency. In particular:
the network access manager coordinated with emergency services
the rolling stock asset manager obtained information of the train’s location and speed to pass onto emergency services
the general manager freight services arranged diversion of the train into Devonport Yard
a Devonport shunter switched point numbers 85 and 88 to divert the runaway train from the main line into Devonport Yard.
Had the runaway occurred outside business hours, the NCO would have been required to respond on their own, and would likely not have been able to achieve the same positive results alone. For example, a shunter would probably not have been available to divert the train into Devonport Yard.
In summary, TasRail’s processes for ensuring immediate network control actions in response to emergencies (such as runaway and authority exceedance) fundamentally relied on the experience and knowledge of NCOs and did not include the provision of procedures, tools and checklists detailed enough to support the effective management of specific types of incidents that require a time-critical response.
Safety-related design and integration problems
Introduction
Given that the RCE entered an unsafe state, and the communication failed mode did not function to stop the train, the ATSB tested the functionality and performance of the generation 3 RCE to assist in determining the possible mode of failure during the runaway sequence. Although the specific mechanism involved in the RCE’s persistent unsafe state was not identified, the testing identified, or confirmed, several problems with the safety-related design of the generation 3 RCE and its integration with related systems.
Unintended activation of locomotive emergency brake
As noted above, during ATSB testing, activation of the RCE dual direction fault interlock was observed to activate the TR class locomotive’s VX vent valves and emergency magnet valve (EMV), triggering an emergency brake response of the locomotive, separate to the RCE. When RCE was in use on a light engine, an emergency brake response could also occur at the initiation of the communication failed mode and in response to rapid applications of the automatic brake. These activations of emergency brake were unintentional and inconsistent with non-remotely-controlled train operations.
An emergency brake application was not recognised by the RCE if the RCE itself did not initiate it. As a result, the RCE attempted to oppose it. Unlike a situation where the emergency vent occurred due to a broken pipe or deliberate driver action, in these circumstances the activation of the locomotive VX vent valves and EMV were temporary. Once they deactivated, the vent stopped and the RCE was successful at recharging the brake pipe, releasing the brakes.
This was true even in situations where the RCE was commanding a service-level brake application at the same time. The RCE would only recharge the brake pipe to the desired level but the train brakes would respond to the pressure rise as an automatic brake release command. Therefore, the level of RCE-commanded brake application would not occur or could release.
The configuration of the RCE air box, which was connected to the brake pipe in close proximity to one of the VX vent valve’s connection at the rear of the locomotive, was found to contribute to the sensitivity of inadvertent application of this VX vent valve. ADE’s understanding that the operation of the VX vent valves were triggered by an electrical command (as opposed to a pneumatic pressure drop) was not correct.
As already discussed, an inadvertent activation of the dual direction fault interlock occurred when the direction controller was changed quickly from forward to reverse at the beginning of the runaway sequence, which resulted in an unintended activation of the locomotive’s emergency brake response.
Reduced recovery timeouts
The timeout periods implemented in the RCE before permitting resets of the emergency brake and penalty conditions were inconsistent with those of the electronic airbrake on the TR class locomotive. The electronic airbrake manufacturer imposed the timeouts to ensure a safe state was reached (that is, the train was stopped) prior to reset. Otherwise, the combination of rapid application and release of brakes at the locomotive would result in uneven brake application due to the inherent delay in braking application and release towards the rear of the train, which could lead to excessive in-train forces and train separation.
The RCE allowed both the emergency stop switch and vigilance penalty resets to be effected after 30 seconds, unlike the 60-second period imposed during non-remotely-controlled train operations. Resets of other RCE states, such as the communication failed mode and control fault mode, did not have a direct equivalence to the electronic airbrake as they were unique to the RCE. Both these modes could be reset without any required timeout while a train was in motion (that is, a running reset).
The appropriateness and effectiveness of shortened or absent time delays was not assessed by ADE when they were implemented. Therefore, the risk of resets being effected before a train had stopped was not quantified and the possibility of it leading to a train separation was not controlled.
Potential persistent unsafe state during initialisation
Analysis of the generation 3 RCE software found that, during initialisation, it had the potential to enter an endless loop (that is, the receiver could ‘hang’) without entering a fault state. Among the conditions that would result in this behaviour were the train brake pipe pressure remaining above 470 kPa, which included fully-released brakes.
In this condition, the receiver would not respond to driver inputs, would not change the status of the locomotive’s previous commands (such as braking and traction), would be unable to continue normal operations, and would not enter the communication failed mode during a loss of communication.
The RCE’s watchdog timer ran through a separate timer loop, so it would not trigger for this type of problem. Dunn (2002) stated:
…gross faults in software structure will usually result in a program halt or crash. In a second category of faults, the program will continue to run but generate incorrect data which can propagate to operator or effectors [control outputs], thereby setting up conditions for a mishap.
However, there was insufficient evidence available to fully evaluate whether this condition actually occurred during the runaway sequence, and the behaviour was not reproduced during testing, either by rebooting the RCE or through imitating the train configuration and control inputs made at the time (as the tests were not specifically designed to evaluate this aspect of RCE function). Furthermore, initialisation only occurs during an RCE reboot, and there was insufficient evidence to establish whether a reboot did occur during the runaway sequence or, if it did reboot, what caused it.
Opposition to emergency brake applications from an external source
ADE advised TasRail that the emergency stop function was the RCE’s ‘primary safety mechanism.’ It was designed to exhaust pressure from the brake pipe on driver command and during some system fault conditions.
In a conventional (non-remotely-controlled) train, the brake systems on all vehicles were designed to respond to any rapid and complete exhaust of air from the brake pipe by applying the brakes, and the locomotive would also automatically remove traction power; both actions were to stop the train and prevent it from subsequent movement. This behaviour was consistent with Australian Standard AS 7510.6:2014 (Braking systems part 6 – train), which required that, in the event of a train separation, an emergency application of the stopping brake occurred on every vehicle of the train (though compliance with the standard was not required).
When the RCE’s emergency stop function was activated, it replicated an emergency brake application on a non-remotely-controlled train. However, the RCE did not recognise emergency brake commands from an external source. The RCE would oppose instances of brake pipe rupture (as occur with a derailment or train separation), and driver operation of the in-cabin emergency brake, by attempting to raise brake pipe pressure.
The RCE’s opposition to the loss of brake pipe pressure would mean that the brake application on some vehicles was less than a full application and the locomotive’s system to remove traction power would not activate if:
a brake pipe rupture occurred a sufficient distance from the locomotive, or
the emergency brake was activated from the driver’s van.
This meant that, in the event a driver did not command an emergency stop using the RCE transmitter prior to a collision or in response to an imminent collision or mid-train derailment, the locomotive could retain tractive effort, driving into the collision or derailment. Similarly, In the propelling configuration (locomotive at the rear), the locomotive could continue powering into heavily-braked wagons, increasing the risk of wheel lift and flange climb derailments.
Effect of communication loss on vigilance and driver-commanded emergency brake functions
Both the RCE’s vigilance and emergency stop functions required an active radio link to operate. That is, in the event the RCE transmitter and receiver unlinked (for example, after turning off the transmitter to reset a fault mode), these functions were not available until communications were re-established.
The locomotive’s vigilance system was suppressed during RCE operation and replaced by the RCE vigilance function. This required a link to the RCE receiver to effect a penalty brake. Because of this requirement, the RCE vigilance function would not activate if the driver became incapacitated while the transmitter and receiver were unlinked.
A driver could apply emergency braking from the locomotive or driver’s van, or with the RCE transmitter. However, most of the cement train operation cycle required the driver to be outside the train (during loading, unloading or shunting). During these periods, the only available means to apply emergency brakes was via the RCE transmitter’s emergency stop switch, and this was not available if the transmitter and receiver became unlinked.
Although the RCE receiver was supposed to stop the train by entering the communication failed mode when unlinking occurred, the resulting brake application was less than that of an emergency stop or vigilance penalty.
Summary
Overall, the generation 3 RCE had several safety-related design and integration problems, which included:
unintended activation-and-release of emergency braking on the locomotive
recovery from an emergency brake application and certain penalty states that was inconsistent with locomotive braking system timeout controls
the potential to enter an unsafe state during initialisation, which was unrecoverable without external intervention
the absence of a means to detect and respond to an emergency brake application from a source external to the RCE
the vigilance and driver-commanded emergency stop functions were unavailable in the absence of an active radio communications link.
All of these problems were readily identifiable after the runaway accident, and should have been readily identifiable before the runaway through more thorough integration and testing. Of these problems, the unintended activation-and-release of emergency braking on the locomotive was likely involved in the occurrence sequence, based on the braking system’s response to the reported driver inputs early in the runaway sequence of events.
In addition, the absence of a means to detect and respond to an external emergency brake application meant that the RCE most likely opposed the initial, unintended, emergency braking application, thereby releasing the brakes. Finally, the potential to enter an unsafe state during initialisation might also have been realised, although there was insufficient recorded data to determine whether this was the case. The other problems, left uncorrected, had the potential to lead to other types of occurrences.
System safety assurance limitations
Introduction
Elements of the RCE design showed that the developer and manufacturer, ADE, made safety an important consideration throughout development and operation of each generation of RCE, and the equipment had many safety functions and design attributes. However, the safety objectives, requirements, and claims were not clear or recorded, so there was no solid foundation for the design of safety elements. As discussed in the previous section, there were also several integration and design issues, which indicated limited design assurance. This section discusses these system safety assurance concepts further.
Use of incomplete locomotive system information
The developer of RCE for a train needs detailed information about the train such as its interfaces, functions, performance and operational hazards. However, limited approaches were made by ADE to TasRail for technical information, as ADE had concerns regarding infringing intellectual property rights of the manufacturers of the electronic airbrake and locomotive. ADE did not approach the manufacturers themselves for this information. Instead, ADE sought technical information from unofficial sources, including from the internet. Furthermore, TasRail were minimally involved in technical aspects of the system’s development.
The information ADE did use was, at least in part, erroneous and incomplete. Without access to vital engineering information about the systems with which the RCE interfaced, integration problems were more likely.
For example, as already noted, ADE’s understanding that the operation of the VX vent valves were triggered by an electrical command (as opposed to a pneumatic pressure drop) was not correct. As another example, ADE incorrectly diagnosed the cause of wheel lock as being due to inadvertent emergency brake application, when in fact the cause later identified by the ATSB was found to be excessive control pipe pressure. ADE’s decision to change the communication failed mode brake pipe pressure reduction to 400 kPa and not 350 kPa did not address the issue, and instead likely increased stopping distance.
Reliance on communication failed mode
For operational reasons, changes were made to the communication failed mode with the introduction of the generation 3 RCE function that decreased the brake application level, and there was no record of an evaluation of the potential effect on safety. Although TasRail tested the effect on stopping distances in April 2018, this testing was not representative of typical operational scenarios (that is, tests were done with an empty train travelling below 15 km/h). .
More importantly, the RCE radio link and the associated communication failed mode were relied upon almost completely for safe operation. All the commands originating from the transmitter passed through the radio link, which included all driver commands, the tilt function and the vigilance function. Without a radio link, a driver could no longer control the train or command an RCE emergency brake application. This made the RCE’s radio link, and its behaviour when the radio link was lost, at least equally important to safety as the emergency stop function.
ADE relied on software intervention when there was a communication failure. The system needed to detect the loss of radio link and then act on it by sending commands from the RCE receiver to the air box and locomotive, which would then vent the brake pipe to provide a brake application and remove or prevent locomotive power.
However, unlike the hardware-induced emergency stop function on the locomotive, which had a ‘default’ or relaxed state of having brakes applied, the RCE’s communication failed mode response relied on fully-functioning software to work. There was always an inherent potential for software flaws to interrupt this process. This resulted in an increased potential for the communication failed mode response to itself fail, as it did during the runaway sequence. It similarly reduced the availability and reliability of the RCE’s driver-induced emergency stop function.
Limitations of pre-delivery testing
The United States National Research Council (2007) stated that ‘it cannot be stressed too much that for testing to be a credible component of a dependability case, the relationship between testing and the properties claimed will need to be explicitly justified.’ This means that tests are only credible if there is a clear relationship between the testing and requirements (part of what is termed ‘traceability’), and there needs to be a record of the manner in which the tests show compliance with the requirements.
Although ADE undoubtedly performed informal ad hoc testing throughout the development and commissioning of the generation 3 RCE (and previous generations), these activities were almost entirely undocumented. There were no detailed step-by-step test procedures, and no pass/fail criteria, so the tests were not repeatable. Where tests were documented, there was no recorded relationship between desired (or required) behaviour and testing methodologies, so there was no way to make an evidence-based argument that the tests provided meaningful information about the safety of the system.
Pre-delivery testing by ADE was limited to bench tests, and ADE specifically noted that several of its bench tests could not be replicated during the on-train testing phase. The tests that were not conducted while fully integrated on the train included the control fault mode (the application of brakes during a control fault being critical for safe operations), and the dual direction fault interlock (which was observed immediately preceding the runaway). The ATSB found that it was possible to test both with the RCE integrated on the train.
The first on-train tests of generation 3 RCE were conducted during commissioning testing on 18 January 2018, but no records existed of these tests or the reason they did not pass. By the time of the second commissioning test conducted in February 2018, a test plan (consisting of a checklist only) had been developed by TasRail. As with the pre-delivery testing, exact objectives were unstated and documentation was minimal. There were no test procedures or descriptions of how the tests were to be performed.
The effectiveness of the onsite testing that was conducted in February 2018 was probably diminished further by a limited understanding of the safety characteristics of the overall system: ADE had limited visibility of the broader system before this, while TasRail did not have any subject matter experts present. As a result, many safety-related aspects of the RCE’s integration were very likely not tested, including the equipment’s response to an emergency brake application from the driver’s van or via brake pipe rupture (both later found to be ineffective).
Limitations in the development process
Although software provides opportunities to enhance the functionality of a system, it can also be a vulnerability. It has been claimed that, in practice, it is impossible to have complete knowledge about a system that incorporates software (Hawkins 2009) or, by extension, any other complex system. This means the safety of such a system cannot be proven with certainty. However, processes and guidance have been developed to enable the safety of a system to be substantiated to a nominal degree of confidence.
Neumann (1994) contended that the development of a complex system with ‘correctness’ requirements (such as safety) requires an extremely principled approach to system development, well beyond what is traditionally experienced in general practice. In other words, the need for a high level of safety assurance demands not only the inclusion of relevant safety features but a structured, principled approach to the development process itself. Doing so helps reduce errors in the design process, adding to safety dependability.
Similarly, Hawkins (2013) stated that the need for traceability is widespread or universal across software safety standards, as are stepwise validation of the software requirements and methods of showing that requirements are satisfied.
These concepts are fundamental to system safety processes such as those outlined in Australian Standard AS 61508 and European Standard EN 50126 (see Appendix B). In general, they provide methods to develop safety objectives, requirements and claims, in order to make an evidence-based argument that the system meets (or does not meet) the objectives.
The generation 3 RCE was not developed using such comprehensive, top-down design principles, so its dependability, and therefore the safety of the broader system, was unproven.
ADE attempted to apply a limited subset of reliability analysis techniques based on AS 61508 to its generation 1 RCE in 1999, but this application was inherently flawed. The highly-structured and fully-documented development process required by the standard was absent, and the necessary groundwork for this analysis, such as a requirements specification, or defined hazards and consequences, had not been established. Therefore, the analysis undertaken had limited value in terms of determining safety.
To a significant extent, ADE did not address critical considerations such as the RCE’s reliance on radio communications, start-up state, software reliability, or the reliability and effectiveness of other safety functions. The assessment explicitly excluded detailed design elements such as ‘part numbers, circuitry structure, [and] detailed software code’ – all necessary for a final assessment to be comprehensive and effective. The assessment also did not recognise that software could have flaws that would not result in the watchdog activating, and it did not include any software assessment even though such processes comprised a significant part of AS 61508.
Most critically, the generation 3 RCE was significantly different to the generation 1 RCE system that was assessed in 1999, rendering the assessment obsolete (even if it were originally applicable), and it was not redone.
Most systems safety methodologies explicitly or implicitly require the production of a significant amount of process and design documentation, which is naturally developed as part of the process; that is, information about the system cannot be fully understood until it is documented. The documentation also serves as evidence for independent verification of the development process and can be revisited when changes later take place.
The design documentation produced by ADE for the generation 3 RCE was not suitable for these purposes—most relevant information was held in informal records and the system’s development and maintenance relied heavily on the developer’s individual system knowledge. Similarly, the development process itself was mostly undocumented and unstructured, and involved just 2 people. According to Hobbs (2020), a lack of ‘intellectual diversity’ among a development team can be problematic.
In the absence of structured system safety assurance activities, ADE’s claim that the RCE would always fail to a safe state could not be substantiated to any reliable degree. ADE did not produce sufficient process and product documentation to allow TasRail to independently verify the safety of the RCE in the context of the broader operational system.
Summary
Although ADE had safety as a design objective and safety elements were included in the RCE, the equipment was not developed using system safety assurance activities appropriate to its application. These limitations in the development process contributed to a range of design and integration problems, as already discussed.
Acquisition and use of remote control equipment
Introduction
TasRail began to consider a replacement for the generation 2 RCE in early 2015 with a formal business case developed in December 2015. It subsequently ordered the new generation 3 RCE from ADE in May 2017.
It is notable that, by the time generation 3 RCE was introduced, TasRail had operated the cement train service successfully using previous generations of the RCE for about 19 years without any accidents attributable to the RCE, and largely without serious incident. In addition, ADE had provided TasRail with assurances of the equipment’s safety integrity. This likely contributed to a high level of confidence in the use of RCE and the overall operation.
Even so, the development and integration of new equipment is not solely the responsibility of the equipment manufacturer. Although ADE undertook the technical challenge of RCE development, TasRail, as the rail transport operator (RTO), ultimately held a duty to ensure that the cement train service (and the RCE as part of the broader system) was safe. However, TasRail had limited further involvement in the development of the system, and there was ultimately no formal definition of safety requirements or verification that the system met a specified level of safety, as outlined below.
Operator involvement in the development process
The generation 3 RCE was a new build specifically for TasRail. Although ADE initially proposed to use an existing product from a third party, this option could not provide some of the functions TasRail had requested, such as dynamic brake, the Railton loading interlock, and conversion to a digital radio on a new frequency. Subsequently ADE proposed manufacturing a new product, which became the generation 3 RCE. TasRail had also been told by ADE in 2015 that the transmitter and receiver microprocessors would change; however, it was unclear whether the extent of the consequences of this change—specifically that it required a complete software rewrite—were fully discussed.
Although TasRail was aware of the above changes to some extent, there were numerous other important changes of which TasRail was unaware. TasRail did not directly ask ADE for information about changes that were being made to the RCE and did not conduct risk assessment activities for the changes that were known.
For example, the level of brake application in the communications failed mode had been reduced by October 2017, but ADE did not inform TasRail until April 2018. Tests of stopping distance were not conducted until June 2018, 4 months after the RCE was commissioned and in use.
Overall, TasRail seemed to not fully appreciate the extent of the change, with a widespread view that the transition to the generation 3 RCE would be ‘like-for-like’. This view was asserted multiple times in documentation:
the original business case in December 2015
the initial impact assessment (IIA) in November 2016
the revised business case in January 2017
discussion with the Office of the National Rail Safety Regulator (ONRSR) in December 2017 in relation to the need for a notification of change.
Conversely, there was evidence showing that the change was not actually like-for-like, including:
requests for multiple changes made by TasRail (including the addition of dynamic brake functionality)
discussions between TasRail and ADE about the need for other changes (including change to radio type and frequency)
identification that driver retraining was needed and provided (this was brief, which may have reinforced TasRail’s perception that the changes were minor; however, major changes can occur while maintaining comparable driver interfaces), along with a new and revised operation manual.
Once the generation 3 RCE was in service, a TasRail staff member raised concern that the communication failed mode brake application had changed. Although this prompted a response within TasRail, it was not a direct challenge to the concept of like-for-like replacement and did not prompt consideration of whether other changes had occurred that had the potential to affect safety.
Even with information available that changes to the RCE were occurring, there was no evidence that the assertion of like-for-like was challenged within the change management process. Consequently, the concept of a like-for-like replacement promulgated through the organisation to become the accepted view, at least until after the generation 3 RCE was commissioned and in service in February 2018.
It is likely that this view, along with a 19-year history of using previous generations of RCE with very few serious problems, influenced TasRail’s approach to the development and post-operational assessment of the generation 3 RCE. Rather than approaching it as a new design, requiring more direct involvement and oversight, TasRail’s approach was more consistent with that of a slight modification to an existing design.
In addition to the accepted view of the change as like-for-like, there were also other limitations with TasRail’s corporate knowledge about the RCE. Within TasRail, a single person had maintained responsibility for management and operation of the RCE from 1999 through to about 2009. While the RCE was still used from then on, there was no nominated person or team responsible for it until 2017. From 2009 on, there was one person nominated for responsibility of the RCE project at a time, with a maximum of about 1 year in this position.
TasRail advised that a shift from individual responsibility to departmental responsibility was consistent with modern practice. However, it is likely that a significant amount of RCE subject matter expertise had been developed over time and then effectively lost in 2009, and again with the change in project managers shortly before commissioning the generation 3 RCE. The limitations in corporate knowledge were likely worsened by the absence of documented RCE specifications, functions, and expectations, and minimal documented information about the project itself. This limited TasRail’s ability to effectively oversee the generation 3’s development and integration from a technical perspective, particularly the safety-critical aspects of its function.
TasRail also applied limited subject matter expertise to the development and oversight of the RCE commissioning tests. The commissioning test plan was reviewed by the TasRail rolling stock assets manager, driver supervisor, and ADE, but there was no detailed independent review of the procedures for completeness and validity by locomotive and train systems subject matter experts. Although TasRail representatives were present during the testing itself, TasRail’s involvement appears to have been largely limited to passive observation. The testing was supervised by a graduate engineer who had started with TasRail the previous month and had little opportunity to become familiar with the locomotive and RCE systems.
Finally, the absence of a structured acquisition process also appeared to contribute to ad-hoc practices in notifying and addressing RCE faults, as well as post-implementation testing.
In summary, TasRail did not fully engage with the development process of the generation 3 RCE from initial design through to commissioning. Associated with this limited involvement it did not understand the extent of design changes that ADE were implementing (and the implications of changes that TasRail were requesting). This reduced TasRail’s capability to prevent, identify and resolve design and integration problems.
Absence of safety requirements
The safety of a ‘system’ (which may be viewed as a broad undertaking, such as the cement train service using RCE, or a subsystem such as the RCE itself) is dependent on a high-level, organisational understanding of safety. When that understanding is limited, the level of attainable safety is similarly constrained.
In general, TasRail had a high-level approach to safety through the application of an organisational safety management system with embedded change management and risk management principles. TasRail also had a long-established conceptual view about how the cement train and RCE would operate.
However, the safety objectives for the cement train service were not explicitly stated. This is particularly important in this case because the operation—and therefore risks—were considerably different to other types of rail operation. This, in effect, made safety an invisible target.
The absence of documented system requirements, including safety requirements, and limited evaluation of safety-related design characteristics, meant that any results of testing would not have been sufficient to make an evidence-based argument about the safety of the system because there was no documented expectation of performance.
In particular, TasRail did not specify a maximum acceptable likelihood that RCE performance could contribute to certain hazardous events, such as a runaway, derailment, or collision with a road vehicle. Had this been done, ADE would have been obligated to prove that this requirement was met to a certain degree of confidence. The process of making this proof probably would have improved overall understanding of the cement train system and RCE subsystem and resulted in a safer design. The tangible evidence it provided would have enabled TasRail’s decision-making process about the broader system to be better informed.
Without TasRail definitions, the safety elements of the RCE design stemmed from ADE’s concept and perspective of what ‘safe’ meant. This was primarily the availability of the emergency stop function, backed by other safety functions and the intent of a fail-safe design. This approach may not provide an optimal level of safety when compared with a carefully considered and explicit definition. Without this information, and with minimal TasRail involvement, ADE had to make assumptions about how the RCE had to be designed, built, and operated for it to be safe, and these assumptions were not formalised or challenged.
The loss of RCE communications, for instance, was recognised as a safety concern: when the RCE detected a loss of radio link for 4 seconds, it would remove traction power and make a service brake application that should bring the train to a stop. However, the RCE response was not based on a formally-defined and evidence-based safety requirement, and was therefore not necessarily optimal. There was no consideration how much braking effort provided minimal risk across the entirety of operations, or whether those parameters should change depending on the train’s load, consist, speed, location or direction. Braking performance tests were not representative of actual operations (there were no tests on a loaded train or at speeds above 25 km/h). There were also no requirements for functions to work under specific conditions, such as the communication failed mode being operable during initialisation, potentially leading to undesired RCE responses under unforeseen combinations of conditions (as happened during the runaway).
Finally, there was no assessment of how critical each RCE and locomotive function was to safety, how dependable these functions were, and whether secondary safety measures existed or were required in the event that an unsafe failure occurred. For example, had there been a requirement for the RCE to detect external emergency brake applications, ADE would have had to implement that function.
Verification of system safety
TasRail’s safety assessment of the RCE and its integration with the overall system was limited. Risk assessments done in 2011 and 2014 largely did not effectively assess the potential for the RCE to fail to an unsafe state. Although the 2011 assessment acknowledged the potential for the RCE to ‘operate abnormally’, it did not contain sufficient detail to evaluate what this meant or verify the effectiveness of controls, and it did not consider causes other than incorrect maintenance or transit damage (such as problems with design and integration). The potential for transmitter brake or throttle control failure, a limited subset of potential causes of an unsafe state, listed a risk control that was later removed (the independent remote stop mechanism), and a safety analysis that was no longer valid after substantial design changes.
After the generation 3 RCE was in service, a risk assessment in relation to a collision resulting from failure of the RCE identified several risk controls, and concluded that the risk was adequately controlled. However, it did not appear to consider their efficacy or reliability, and the stated risk controls were actually not reliable. More specifically, in terms of the cited risk controls:
Driver competency could not always address an RCE failure; for example, when the driver was not on board the train.
Several RCE and train functions were cited without consideration of their effectiveness. Some would not work with a lost radio link or were being routinely reset without diagnosis of underlying cause.
The driver’s van emergency brake was largely ineffective when tested by the ATSB, and had the potential to increase the risk of derailment in some situations.
The generation 3 RCE itself was seen as reducing risk without consideration of the potential for hazards generated or changed by the new equipment.
Several further changes were made to the generation 3 RCE after commissioning was complete. There were no formal risk assessments for any of these changes. Most were implemented without subsequent re-testing by TasRail or ADE and, when re-testing did occur, there was no record kept of the tests conducted or outcome. In fact, the many changes introduced with generation 3 diminished the value of confidence built from in-service experience of the previous RCE generations.
More broadly, TasRail did not perform or require detailed system analysis in order to verify its safety. To ensure that a system is dependable, engineers need to understand how faults or software errors can appear, and how they can manifest in all operating conditions and modes. Different techniques can be used to estimate or predict the likelihoods of different kinds of failures, and evaluate the potential effects in a technical and operational sense (Dunn 2002). Similarly, potential undesirable outcomes can be traced back to a range of potential causes to identify areas of concern.
When the results are combined to form a comprehensive view of the overall safety of a system, it can culminate in a ‘safety case,’ described by the United Kingdom Ministry of Defence (2007) as:[88]
A structured argument, supported by a body of evidence that provides a compelling, comprehensible and valid case that a system is safe for a given application in a given operating environment.
For the argument to be compelling and valid, all feasible safety concerns should be addressed by the safety case. Neither TasRail or ADE produced a safety case (or equivalent argument) for the use of the RCE, and important elements of the application and environment were not considered in its design. The use of systems safety assurance activities would have been much more likely to identify that, in some circumstances, RCE failure modes could result in an unsafe state, or provide external risk controls if the risk could not be effectively managed another way.
Most importantly, there was no record of the degree to which safety-critical functions were checked for effectiveness at any point throughout the generation 3 RCE system’s development and commissioning. Given the nature and duration of the tests, they were very likely basic functional tests, which are mostly uninformative about the dependability of the system as a whole, particularly when software is involved (National Research Council 2007). For example, there were no documented assurance activities to prove that the ADE-assessed critical safety function (that is, the emergency stop) was adequately reliable.
Summary
In summary, TasRail commissioned the manufacture of, and continued to use, safety-critical locomotive RCE without systematic assurance of its safety. This was because TasRail did not:
fully engage with the development process from initial design through to commissioning, or understand the extent of design changes associated with the introduction of the generation 3 RCE (which reduced its capability to prevent, identify and resolve design and integration problems)
explicitly identify and impose safety requirements
verify that the overall system met a specified level of safety.
In other words, TasRail relied on ADE to provide a product that was fit for purpose without seeking independent clarification of the claims made by ADE regarding the engineering standards met by the RCE or the safety validation process used. As a result, the RCE was provided and used for the TasRail cement train operation without a compelling argument for its safety.
Safety implications of remotely-controlled trains
Introduction
As well as with the design of the RCE itself, there were also additional safety aspects to be considered when using remotely-controlled trains. TasRail had utilised RCE to operate its cement train service in a broadly consistent way since about 1999. RCE was the primary means of operation at all times, being used to control the train even when the driver was stationed in the locomotive cabin.
This section discusses how the use of RCE introduced key differences when compared to conventional train operation (which required a driver directly operating the controls of a locomotive positioned at the front of the train at all times). The change in control system added significant complexity to the control of the train, and presented additional challenges that needed to be managed.
Potential for remote control equipment to fail to an unsafe state
A driver operated remotely (off the train) for the majority of the cement train’s service cycle, during which the in-cabin emergency brake controls, as a means to place the train in a safe state, were not available. Therefore, for the majority of the time, it was critical that the RCE remained responsive to driver commands or would always fail to a safe state.
TasRail’s cement train system depended heavily on the RCE being fail-safe, and it was designed to have that result; that is, so that any hardware or software failures would always result in a safe state. However, uncertainty is an unavoidable consequence of having computers (or any complex design) as parts of hazardous but reasonably safe environments (Littlewood and Strigini 1995). There is always the potential for a complex system to fail in unpredictable ways; no system can ever be guaranteed to work acceptably all the time and it is impossible to predict all the sources of failure (Neumann 1994). Therefore, the potential for the RCE to enter unsafe failure modes was always present, simply as a result of its computer-based architecture.
It is important to account for the potential for a system to operate in (or fail to) an unsafe state in the design of how equipment integrates into the overall system and add or adjust external elements if the safety risk remained too high; that is, adjust the architecture of the broader system to accommodate the potential for critical subsystems to fail (Dunn 2002). At the time of the runaway accident, there was no backup or redundant system to assert control in the event of an unforeseen RCE failure mode, and the RCE itself did not have a proven level of system safety integrity.
An alternative approach was taken in the 2005–06 series of reports by the United States Federal Railroad Administration (FRA), which recommended limits on locomotive power, train length, speed and grades for remotely-controlled trains, effectively constraining their use to yards and short distances.
TasRail had implemented a secondary safety system in response to the generation 1 RCE entering an unsafe state in February 2010. This was an independent remote stop mechanism to remove traction power and apply emergency brakes, which could be used in case of RCE failure. Even though the independent remote stop mechanism did not have an assured level of reliability, it provided a risk control that was functionally independent of the RCE, lessening the reliance on RCE being fail-safe.
The generation 2 RCE had a safety feature—the dual direction fault interlock—designed to address the specific failure mode encountered in February 2010. When the generation 2 RCE was introduced into service, the independent remote stop mechanism was removed, even though the dual direction fault interlock only addressed a subset of potential causes of the system entering an unsafe state. This indicates that neither ADE nor TasRail recognised at the time that there could be other reasons for the system to enter an unsafe state.
Analysis of recorded data indicated that the generation 3 RCE entered a potentially unsafe state at Caroline Creek in May 2018. During this event, the RCE ceased responding to driver commands and the train’s brake pressures were changing in an uncontrolled manner that could have led to the brakes being released. This showed that the fail-safe design intent was not dependable. In safety terms, this event was likely a precursor to the Railton runaway accident.
In summary, the generation 3 RCE did not have high levels of proven system dependability, either through extensive in-service experience or the application of structured system safety assurance principles, so there was a realistic potential for it to fail to an unsafe state. However, there was no design redundancy or secondary means to assert control in the event of an RCE failure condition that resulted in it being unresponsive or making spurious commands.
Suppressed or absent safety functions
Vigilance control and overspeed protection functions were safety features present on the TR class locomotive. However, they were suppressed when it was configured as a trailing locomotive (in ‘trail cut-out’ mode), as was required when the RCE was in use.
As previously discussed, although a similar vigilance function was active on the RCE transmitter when the train was under RCE control, it relied on a communication link between the transmitter and receiver to function effectively. The penalty brake application induced by a vigilance penalty, and the time delay before a reset was permitted after a penalty occurred, were also different (less) than for the locomotive system.
With regard to overspeed protection, a similar function, a GPS speed restriction, was offered by ADE. However, it was not requested by TasRail or implemented in the delivered version of the generation 3 RCE.
Certain indications and controls that were normally available to the driver in non-remotely-controlled operations were also not available when the RCE was in use. Some were not available when the driver was operating from the driver’s van or outside the train locomotive cab, and others were not available at all. These included:
The brake pipe flow indicator was not available at all, as the locomotive’s electronic airbrake was set to trail cut-out mode for RCE operation.
Locomotive brake cylinder pressure was not displayed when operating from the driver’s van or outside the train locomotive cab. Instead control pipe pressure was displayed as a proxy on the RCE transmitter and a light would also illuminate. However, this did not provide an accurate indication of the locomotive’s actual brake cylinder pressure (such as during automatic brake applications where the brake cylinders may be charged without the presence of control pipe pressure).
The emergency engine stop button was not available when operating from the driver’s van or outside the train locomotive cab. There was no alternative method available for the driver to stop the locomotive engine when operating from these locations.
Non-recording of key data parameters
The use of recorded data is often a critical aspect of incident and accident investigation, and the data available from the locomotive’s event recorder enabled the ATSB to reconstruct much of the runaway and derailment sequence.
However, no RCE data was recorded, which limited analysis to that which could be inferred from locomotive responses only. This contributed to the ATSB being unable to determine or replicate important aspects of the runaway sequence, such as RCE modes and commands and identify the fault state that the RCE had encountered.
As discussed previously, the use of RCE represented a change in the train’s control system when compared to a non-remotely-controlled (conventional) train. The driver operated the RCE transmitter rather than the locomotive controls, and the RCE had its own system for effecting brake commands. This added additional complexity to the control of the train as commands were being relayed between systems (between the transmitter, receiver, air box and locomotive).
The increased complexity meant that additional monitoring was needed in order to interpret system command and response behaviours. However, TasRail did not add to or modify the data recording capability for the remotely-controlled cement train. The recording configuration was the same as that used on TasRail’s non-remotely-controlled trains when hauled by a TR class locomotive.
Although recording of key parameters would have assisted further investigation of the runaway accident, it also had the potential to assist TasRail and ADE in identifying issues with the RCE behaviour encountered prior to the accident, during both testing and precursor events, such as that which occurred at Caroline Creek in May 2018. TasRail later reported that recording of parameters relating to RCE would have assisted in both fault finding and rectification during the implementation of generation 3 RCE. It is likely that access to sufficient recorded parameters would have helped TasRail and ADE to have identified that the RCE had failed to an unsafe condition, and either addressed the underlying reasons for it or implemented alternative means to ensure safety if the RCE entered this state in the future.
It was apparent that TasRail was aware of the limitations of not recording key parameters on the generation 3 RCE, and had initiated enquiries to obtain a data-recording capability after commissioning the RCE. However, it had not addressed the issue by the time of the accident. Further discussion about the importance of and requirements for recording RCE functions is provided in Recording of remote control equipment functions.
Risks associated with propelling operations
Derailments can occur from wheel lift and flange climb, due to insufficient vertical wheel load or increased lateral force to the rail head. To prevent these outcomes, slack within a train is gradually ‘stretched out’ (draft force) from a locomotive hauling rolling stock behind it. Conversely, buff (compressive) forces associated with propelling operations increase both of these forces and the risk of derailment.
Although propelling at main-line speeds is an uncommon practice, another motive power configuration with the potential for similarly-increased buff forces, called distributed power,[89] is widely used (in Australia and internationally). The RISSB Code of practice: Distributed power freight trains (July 2018) recognised that distributed power operations have the potential to ‘generate train forces exceeding safe limits’ (both compressive and tensile) and recommended that operators intending to use distributed power conduct analysis to manage these forces.
When TasRail operated its cement trains in a propelling configuration, the driver had access to an emergency brake handle that could vent the brake pipe from the driver’s van, initiating a brake application. Use of this handle was an appropriate action to take in some emergency situations, such as when risk of a level crossing collision existed or during an RCE fail-to-unsafe event. However, tests conducted by the ATSB showed that, if the driver’s van emergency handle was used while the locomotive was under power, there was potential for the locomotive to continue pushing against the brake application, further increasing the buff (compressive) forces in the couplings between wagons. This risk was also present if a derailment or collision occurred and an RCE emergency brake command was not immediately enacted.
TasRail advised that the cement train had never derailed in almost 20 years of remotely-controlled train operations, and of course the 21 September 2018 runaway and derailment was not due to the propelling configuration. However, the risk of derailment was increased due to the use of propelling compared to that which existed for non-remotely-controlled train operations. TasRail did not formally assess this risk or appear to have considered its implications. Further, there was no formal instruction given to drivers regarding the unique train handling risks associated with the cement train’s propelling configuration.
Summary
The use of RCE when operating trains provides advantages but also provides a range of potential challenges that need to be identified and managed to ensure the safety of operations. TasRail had conducted 2 risk assessments related to remotely-controlled train operation in 2014. However, these relied on the RCE as a reliable ‘fail safe’ risk control and did not consider risks that the use of RCE introduced.
By the time of the runaway accident in September 2018, TasRail had not formally assessed the risk of many of the safety implications presented by the use of RCE with the cement train service. It had used several generations of RCE for about 19 years to perform this task, so it is likely that over this time the unique application of this technology became normalised without underlying hazards being thoroughly explored. It was only when generation 3 was introduced that some of these inherent limitations became more pronounced.
In summary, although TasRail had used RCE for about 19 years, it did not identify or fully assess the safety implications of remotely-controlled train operations, or those of TasRail’s specific implementation. These included the:
potential for remote control equipment to fail to an unsafe state
suppression or absence of some safety functions of the TR class locomotive
nonrecording of key data parameters to facilitate effective fault and incident analysis
increased risk of derailment as a result of motive power located only at the rear of the train.
Change management limitation
Introduction
The need for a new generation RCE was identified in early 2015 and it became clearer by the end of 2015, when TasRail were informed that the generation 2 RCE’s radiofrequency spectrum allocation was required to be vacated by April 2017. At this time, ADE advised TasRail that the current-generation RCE was not compatible with the new radiofrequency spectrum allocation.
TasRail recognised that the development of generation 3 RCE was a change and initiated its change management process, including developing multiple business cases and an initial impact assessment (IIA) in November 2016. A revised business case was submitted in January 2017 and the equipment was ordered from ADE in late May 2017. There were subsequently unforeseen delays during development, requiring repeated extension requests, and it is reasonable to expect that the project team experienced significant time pressures to finalise the RCE acquisition once the process did commence.
In addition, there were a number of problems associated with TasRail’s application of its defined change management processes, and also the design of the change management process itself. These are discussed in the following sections.
Management of change to generation 3 remote control equipment
The IIA was the main driver of the change management process, assessing the project level at commencement and determining the process to be followed. It also assessed the impact and size of the change, including potential impact to safety.
The IIA assessed the project as a ‘significant change’. Although the project was first assessed as a P1 change, the IIA was revised in January 2018 to reclassify it as a P2 change. The reason for this was not documented, and contradicted TasRail’s general treatment of the change as like-for-like, but it was probably appropriate given the potential negative impact to safety.
TasRail’s processes dictated that such projects carry out 33 documented activities. On the IIA, the record of project management assurance requirements and approvals (indicating that a project is formally approved from the outset) was blank. Overall, of the 33 documents that should have been produced during the project, only a third of the required documents were produced and almost half of these were incomplete. There were no records of executive approvals for the 6 gates that nominally required them, yet the project was able to continue.
In addition, although TasRail’s corporate governance and risk management process required a reassessment of both risk and its controls when a ‘significant change’ was identified, this was not done. Other safety-related steps not undertaken during the project included:
change management plan
stakeholder engagement and communications
technical investigations / preliminary designs
acceptance for testing
acceptance for operations.
At project commencement, TasRail’s project and change management processes were in draft form. They were not fully implemented until the end of July 2017 while the project was underway. This may have had a bearing on TasRail’s ability to monitor and control the project during a period when these processes were being applied in practice for the first time.
In summary, the actions and documents required for each phase in TasRail’s change management process either were not conducted, were incomplete, or did not contain the required formal approvals. As a result, important safety-related steps were not completed.
Assurance of change management activities
The generation 3 RCE project was able to proceed largely outside of TasRail’s change management process primarily because the checks and balances within the broader system did not detect that this was happening. Specifically:
There were no mechanisms by which non-completion of activities, documents or approvals would prevent progression of the project.
TasRail had an audit program to ensure compliance, which this was limited to 12 selected projects per year. The generation 3 RCE was not chosen to be audited, and other audit requirements did not apply to this system.
Funding for the equipment was (at least in this case) not dependent on the process having been followed.
In essence, the change management processes only nominally required that certain activities were conducted; and the mechanisms in place to detect when activities were not conducted and halt the project or alert people who could intervene were ineffective.
Identification of relevant safety assurance activities
TasRail had engineering design, verification, validation, and acceptance procedures, including those specifically for safety-critical items of rolling stock that would be considered relevant to a technical project such as the development of the generation 3 RCE. The procedures included requirements that technical specifications be developed and risk assessment and failure modes and effects (FMEA) analysis be performed. They also provided guidance on the level of technical expertise that should be involved.
However, there were no links to these procedures from elsewhere within the TasRail system of documents. In particular, there was no requirement to either follow or consider these procedures from within the change management process. Therefore, even if the change management process been fully followed for the project, the relevant safety and engineering assessment activities (with a significantly greater potential to prevent or identify safety-related problems) would not necessarily have taken place. There was also the potential, albeit slight, that the relevant design assurance and acceptance activities would not be conducted in other projects, including the acquisition of new locomotives or other types of rolling stock.
As an example of where the absence of links between documents may affect the effectiveness of the change management process, there was no requirement or prompt for the person who completed an IIA to consider TasRail’s definition of ‘safety critical’ (as written in the engineering processes) when determining whether a negative implication to safety existed.
Further, TasRail’s change management process did not consider whole-of-life system management; that is, from initial planning and design to system decommissioning. This may have prevented the limited monitoring and review of risk associated with ongoing operation of generation 3 RCE after it was commissioned and had the potential to affect other projects as well.
Identification of potential safety impacts of change
Although the generation 3 RCE project could be considered small when compared with some of the projects undertaken by a rail operator (such as major infrastructure changes or the introduction of a new locomotive type), in safety terms a small project can still have a substantial impact. Even though the project was ultimately classified as significant (P2), the defined processes would not always result in the appropriate classification on the basis of safety in other circumstances.
The template IIA form directed assessors to classify a change as small or significant on the basis of 9 questions. The only question that had direct relevance to safety was whether there were ‘any negative safety implications associated with the initiative’. In this case, the assessor answered ‘no’ and the project was classified as significant on the basis of other factors such as cost.
Such a question is inherently subjective, but more so in the absence of clear definitions. The significance of a proposed project with minimal strategic and business impacts could hinge on an individual’s understanding of its ‘negative safety implications’ with no guidance as to what this means. As stated previously, the definition of ‘safety critical’ was in a separate document not linked to the IIA or other change management documents.
Also, the IIA form did not require an assessor to list or otherwise identify specific hazards, or to carry out any safety analysis activities beforehand. The change management process also did not require a hazard or risk identification activity prior to completing an IIA. This could lead to the incomplete consideration of hazards.
In addition, a change that did not qualify as significant under any of the other questions could then proceed as a small change with minimal control or oversight, even though the potential to impact safety could still be substantial.
The questions and guidance for determining safety of a significant change were similarly limited, potentially leading to under-classification. However, this would normally be less of a problem because all ‘significant’ projects required safety-related activities like risk assessments and risk action plans.
Summary
In summary, TasRail had a detailed change management process in place, and had documented that the project to develop the generation 3 RCE was a significant change. However, the project, a significant change with unique operational risks, proceeded without many of the activities required by TasRail’s change management process being completed. Furthermore, even had the defined process been followed, TasRail’s systems for management of change had a limited capability to:
assure pre-determined activities, approvals, and documentation were completed throughout progression of a change
identify the need for relevant safety assurance activities
assure the determination of whether a change had the potential to impact safety.
Processes for fault tracking and analysis
Introduction
It is obviously most effective and safer for problems with the design and integration of a system to be identified and managed prior to its implementation into normal operations. Nevertheless, even after implementation, there are opportunities available to identify and manage problems.
This section discusses TasRail’s management of fault reports and other issues that arose during the first months of generation 3 RCE operation (that is, after it was commissioned in February 2018 and prior to the runaway in September 2018).
In-service tests
There was no formal process for TasRail drivers to verify RCE safety functionality before continuing operations after an RCE failure occurred. ADE did not stipulate a requirement, apart from after a cold start, and drivers advised the ATSB that tests were generally only conducted after a prolonged locomotive and train shutdown had occurred.
Although periodic functionality checks existed, the RCE was not required to be checked in a similar fashion to a locomotive, although it replicated many locomotive commands. Further, in-service periodic testing of RCE safety functions, such as the RCE transmitter emergency stop switch and vigilance and tilt functions, were not performed by drivers, nor did TasRail require them to be.
Fault reporting and follow-up
Although multiple TasRail staff were involved in addressing reported generation 3 RCE faults, the extent of their responsibility in this regard was unclear and there did not appear to be anyone identified to lead the fault report analysis and management process.
Exacerbating the limited asset ownership and oversight was the absence of a formal process for reporting or addressing faults specifically related to the new RCE. More specifically:
There was no procedure or dedicated form for drivers to record generation 3 RCE faults.
A shift form partially tracked RCE faults, but only in relation to time delays.
Drivers advised the ATSB that, unless an RCE fault was persistent or caused a significant time delay, it was unlikely they would report or log the fault.
Although the network control officers (NCOs) would log emergency alarms from RCE failures and RCE faults reported to them by drivers, there was no defined escalation process.
It is certain that drivers encountered more faults with the generation 3 RCE than was formally reported or recorded. TasRail had some awareness of under-reporting of RCE faults; however, it is unclear how, or whether, it addressed this problem.
Further, of those faults that were reported, there was evidence of misdiagnosis or a lack of information available to define what the faults were and why they were being encountered. For example:
The control fault mode response had been changed, which made its response similar to the previous generation’s communication failed mode response. This change, and the visual similarity between how the two fault modes were displayed on the transmitter (CNF or COMF), introduced the potential for confusion between the 2 modes, leading to misdiagnosis by drivers and other staff.
The origin of control fault mode occurrences, where identified, was not investigated or determined.
Uncommanded emergency brake applications, similar to that observed during the runaway on 21 September 2018, were reported by drivers. These were not attributed to any known fault or penalty condition.
Fault modes that required a cold start of the RCE receiver to reset were reported. However, there were no documented fault conditions that required a cold start to clear. This indicated that an unidentified failure mode was occurring.
Even though the type, origin, and cause of some of the reported faults and recovery actions were unknown or unexplained, further investigation before operation recommenced was not common.
When faults were further investigated, the limited information available (both immediately available and recorded) was a barrier to the identification, analysis, diagnosis and thereby correction of reported RCE faults by both TasRail and ADE. Problems with the recording of relevant RCE data are discussed further in Recording of remote control equipment functions.
In addition, other limitations also existed in the follow-up of RCE fault reports, separate to the limited data availability. For example:
Only some logged RCE faults were forwarded by TasRail to ADE to review. Outcomes from these reviews were not formally documented.
TasRail specifically consulted with ADE regarding driver reported safety concerns, at which point ADE assured TasRail that the RCE met fail-safe requirements. However, it did not support this claim with evidence and TasRail did not attempt to verify the claim.
Although TasRail recorded at least 100 remote-control failure events (in addition to much more common communication failed mode events) from January to September of 2018, only one was identified as a safety concern and captured in TasRail’s risk management system. Even then, this was done due to the issue of limited trackside access, not a concern over the safety of the RCE during the failure. That events were not treated as safety concerns was further indication that TasRail believed that the reported failures resulted in a safe state and that the RCE was a failsafe system.
Prior fail-to-unsafe event
One reported fault event occurred while the cement train crossed Caroline Creek in May 2018, 4 months before the runaway involving train no. 604. Although TasRail requested ADE to analyse this event, it is unclear what response ADE provided to explain it. ADE was unable to explain the cause of the event to the ATSB.
Analysis of the event conducted by the ATSB indicated that the RCE had not enacted the behaviour expected for the documented fault modes and had therefore not failed to a safe state. The unknown state that the RCE entered was only recovered after an uncommanded locomotive engine restart resulted in a receiver reboot. The analysis concluded that both the behaviour of the generation 3 RCE while a fault state was present, and that external systems were required both to maintain locomotive brakes and cold start the RCE receiver, were indications that the state the RCE was in had the potential to be unsafe.
Although the available data was not sufficient to determine why the RCE had entered this fault condition, the responses recorded during this event were not consistent with the intent of the RCE as having been designed with the objective of being fail-safe. This was not addressed by either TasRail or ADE and the generation 3 RCE was allowed to continue in service.
Summary
In summary, TasRail did not have a reliable process to systematically identify, track and analyse reported faults on its remotely-controlled trains in order to identify their potential safety implications. The repeated instances of the RCE ‘failing’, albeit without consequence, were not acknowledged as indicators of a potential safety problem: numerous operational issues may be indicators of design or process issues, which can indicate latent safety flaws.
Overall, the absence of a formal oversight structure that could monitor safety trends in relation to repeated reported faults significantly limited TasRail’s ability to identify and manage problems with the generation 3 RCE prior to the runaway.
Notification of change to the regulator
Notification of change
Accredited RTOs are required to notify ONRSR of certain types of changes to their operations. Notifications provided ONRSR with visibility of changes that could affect safety, allowing ONRSR to initiate regulatory activity if needed.
Through published guidance material, ONRSR encouraged RTOs to contact them if clarification was needed as to whether a given change was notifiable. TasRail’s risk and compliance manager did so for the generation 3 RCE project in December 2017, prior to RCE testing and commissioning.
As already noted, the concept that the generation 3 RCE project was like-for-like was an accepted view within TasRail. It is therefore reasonable that the compliance manager, who only had limited involvement in the project, also held this view when they described the change as like-for-like with the addition of a dynamic brake to the ONRSR rail safety officer.
The ONRSR rail safety officer made a record of this conversation that is consistent with an understanding that it was simply an identical replacement unit, or at most a very minor change, repeating the term ‘like-for-like.’ On this basis, the ONRSR rail safety officer advised the TasRail manager that it was not a notifiable change.
Consequently, associated with the limited internal communication about the extensive changes from generation 2 to generation 3 RCE, TasRail did not submit a formal notification of change for generation 3 RCE to ONRSR. This limited the regulator’s visibility of the change and, as a result, regulatory oversight of the project was not initiated. It is likely that, had ONRSR had greater awareness of the extent of the change, its oversight activities would have helped both ONRSR and TasRail identify limitations with TasRail’s change management processes (discussed in Change management limitations).
Guidance on the regulatory requirements for a notification of change
Under Australia’s co-regulatory rail regulation framework, RTOs are granted ‘operational flexibility to establish and implement standards, rules and methods of operation’ to meet regulatory requirements. ONRSR provided legislative interpretations, guidance and compliance expectations to ensure consistent application of the regulatory requirements without constraining this operational flexibility.
Guidance on the change notification process included an overview of the process, brief interpretations and examples for some of the 12 notifiable items under regulation 9(1)(a), and a recommendation that RTOs contact ONRSR for clarification where required. However, overall there was minimal guidance information provided for interpreting what types of changes required notification, including for changes related to item 3 (‘a change to a safety critical element of existing rolling stock’), either for RTOs or for ONRSR’s rail safety officers.
Three important terms were used in the item 3 requirement:
change
safety-critical
element of existing rolling stock.
However, these terms had the potential to be interpreted differently by different parties, particularly given the minimal guidance information provided by ONRSR.
More specifically, the term ‘change’ alone does not convey the extent of functional, physical, or software changes for which a notification of change is justified. It could mean:
a replacement item that is identical to the original
an item that is very similar to the original in terms of its function and mechanism of operation
an item that had identical functions but different mechanisms of operation, or
an item that could be used to functionally replace the original, but had additional or changed functions and/or mechanisms of operation.
The differences between the new and replacement item could vary in importance and extent, and some changes could require modifying procedures or retraining of operators (in terms of how to use the equipment and/or underlying system knowledge). Moreover, even a small change to one function could inadvertently affect other functions in unexpected ways, particularly in complex systems. It is therefore important to be clear about the nature and extent of changes that require closer attention.
The term ‘safety-critical’ also had the potential for different interpretations. In the absence of a consistent definition, there is a potential for ‘safety-critical’ to be interpreted to refer to any item that could result in catastrophic consequences if it failed, rather than something that has a non-trivial contribution to safety. ONRSR provided guidance (in the Notification of change fact sheet) indicating the term meant equipment ‘whose failure could substantially contribute to a major accident.’ Dunn (2002) stated:
The term “safety-critical” usually brings to mind nuclear plants, oil refineries, airliners, and other high-visibility applications where loss of safety can kill or injure in large numbers. ..[but] most safety-related system applications do not fall into the high-visibility category.
ONRSR’s guidance was consistent with this view, identifying safety-critical elements as those ‘whose failure could substantially contribute to a major accident.’ However, the same guidance stated that these elements are identified by the RTO’s safety management system, and ONRSR separately advised the ATSB that there was ‘scope for an operator [RTO] to interpret this [term] in the context of their own railway operations.’
TasRail defined ‘safety-critical’ as ‘any element whose failure or malfunction may result in death or serious injury to people, or loss or severe damage to equipment or environmental harm.’ A minimum severity of consequence would therefore needed to meet TasRail’s definition, but not necessarily a major accident.
The former Transport (Rail Safety) Regulation 2010 (Qld) applied the term to ‘any part of the rolling stock that could cause, or affect the impact of, an accident or incident if the part failed to operate properly.’ ONRSR’s interpretation supplied to the ATSB during the investigation was similarly broad, using the term ‘safety critical’ to refer to the potential for the equipment to cause, contribute to, or fail to mitigate an unsafe outcome, regardless of the seriousness of that effect. These broader definitions are more likely to capture elements that are safety-related even if their failure might not result in catastrophic outcomes.
Finally, the term ‘element of existing rolling stock’ was not clear for the purpose of the regulation. During the investigation, ONRSR advised the ATSB that, as the RCE controlled the movement and operation of a locomotive, it was to be considered an ‘element of the rolling stock’. The TR class locomotive was obviously rolling stock (a ‘vehicle that operates on or uses a railway’), and the components of the locomotive were ‘elements’ of existing rolling stock. However, although it is a valid argument that the RCE is a piece of equipment that attaches to and controls a locomotive (vehicle), there was the potential for RCE to not be viewed as a literal component of the locomotive.
ONRSR’s internal Notification of change procedure, which provided context for rail safety officers when giving effect to regulation 9(1)(a), did not provide interpretations or guidance for the application of these 3 terms (‘change’, ‘safety-critical’, and ‘element of existing rolling stock’). Ultimately, in the absence of clarifying guidance to aid in the interpretation of these terms, there was no assurance that individual officers would interpret the terms in a manner that was consistent and optimal for the regulation’s (and ONRSR’s) safety intent. This increased the potential for different rail safety officers interpreting the legislation differently and giving conflicting or incorrect advice to RTOs.
Australian Standard AS 4292.3 (Railway safety management, part 3: rolling stock) provided guidance on what should be considered ‘safety critical items’ on rolling stock, but this standard has been withdrawn and the information was not reproduced among the RISSB products that superseded it or any guidance material provided to industry by ONRSR.
With regard to the generation 3 RCE, as it was the primary driver control interface, a failure could cause (or affect the impact of) an accident. Therefore, the RCE must be considered the type of equipment for which a change would be notifiable to ONRSR.
The ONRSR rail safety officer recorded the change as being ‘a like for like replacement of the existing unit which has been in operation for some time and due for routine replacement.’ This description is consistent with that of a routine equipment swap, not normally considered a significant change, and the officer did not know of any design or functionality differences with the previous equipment. Therefore, it is likely that the rail safety officer’s understanding of the nature of the change was a key consideration in their decision not to require a notification of change in December 2017, rather than whether the equipment was safety-critical or an element of existing rolling stock. Given that this type of change would not normally require notification to the regulator, it is unlikely that the limited guidance provided by ONRSR to TasRail and the rail safety officer had an effect on the outcome in this instance.
In summary, the guidance provided by ONRSR about the requirement to submit a notification of change included limited detail about the extent or type of changes that necessitated a notification. In addition, with regard to ‘a safety critical element of rolling stock’, it did not provide detail with regard to the interpretation of ‘safety critical’ and the applicability to equipment that may not be inherently part of rolling stock (such as RCE). This situation was exacerbated after 2018 when some of the limited guidance provided by ONRSR for interpreting notification of change requirements (including a definition of ‘safety critical’) was removed in the transition from published guidance documents to an online portal.
Recording of remote control equipment functions
As discussed in Non-recording of key data parameters, the absence of recorded RCE data led to difficulties in fault finding and rectification throughout the use of the generation RCE and limited the determination of important aspects of the 21 September 2018 runaway sequence. It also potentially prevented TasRail and ADE from developing a full understanding of the Caroline Creek event and identifying strategies to minimise the likelihood of a further fail-to-unsafe event.
The use of an event recorder on all TasRail trains, including the cement train service, was required by a condition in TasRail’s notice of accreditation issued by ONRSR. The use of event recorders was not mandated by any other means, and the use of a condition on a notice of accreditation for this purpose was not common.
The condition on TasRail’s notice of accreditation stated that all ‘leading locomotives or other vehicles used as the leading vehicle’ were to be fitted with an event recorder. The reason why the condition was originally included by the Tasmanian rail regulator, and whether it was specifically introduced for the operation of remotely-controlled trains, could not be determined. ONRSR advised the ATSB that ‘an event recorder fitted anywhere in the consist, that is recording the key parameters’ would meet the intent of the condition. The question of whether the RCE constituted a ‘leading locomotive’ could be made on the basis of physical characteristics (the RCE is not a locomotive, and the locomotive was not always at the head of the train) or functionally (the RCE functions as a leading locomotive from a control perspective), but the distinction is not critical. An event recorder needs to record enough information to reconstruct enough of an accident or incident sequence to determine what went wrong; in the case of TasRail’s cement trains using RCE, this core purpose was not met.
ONRSR did not further define or explain what it considered ‘key parameters’ and TasRail also did not define what parameters were required to be recorded. Effectively, the condition could be met without recording the key parameters that would fulfil the core purpose of an event recorder.
An Australian standard for rolling stock event recorders (AS 7527:2015) described event recorder requirements and the parameters an event recorder should monitor to capture a minimum set of appropriate data. When compared to this standard, the data recorded during operation of TasRail’s cement train service under RCE control did not include the following:
controls being directly operated by the driver
the vigilance function when performed by the RCE transmitter
all changes to the operational status of a remotely-operated vehicle
all control messages and status messages received and transmitted by a remotely-operated vehicle.
In summary, although ONRSR required TasRail to have an operating event recorder fitted to its trains, this requirement did not necessitate that key parameters of the RCE (and the driver’s operation of the train) were recorded. Had the condition been more specific or functional in nature, it could have led TasRail to introduce a more effective recording capability for its RCE operations. Nevertheless, more consideration by TasRail of the contents of AS 7527 when introducing the generation 3 RCE (or before) would also have helped to address the situation.
System safety in the Australian rail industry
As outlined in this analysis, there were numerous latent safety concerns with the use of the ADE RCE in TasRail’s cement train service. These largely arose through the limited application of safety management and safety engineering methodologies to a complex system. While the principles of safety management systems are widely applied across the rail industry, this and other recent ATSB investigations have found that safety engineering is sometimes not applied where it is needed.
The Australian Rail Safety National Law (RSNL) required organisations that procured or supplied rolling stock to ensure safety ‘so far as is reasonably practicable,’ including provisions for safety testing of equipment. However, although testing may be sufficient to ensure safety when systems are relatively simple, there are challenges in assuring the safety of software-based and other complex systems, in part due to difficulties in modelling complex interactions between system elements.
Key areas where equipment developers have had difficulty include specifying the requirements, designing and implementing a system to achieve the dependability requirements, and gaining confidence that the goals have been attained (Littlewood and Strigini 1995). From a practical perspective, it is difficult to decide the type, quality, and quantity of evidence required and the methodologies used to develop arguments about safety (Hawkins 2013). There is a considerable body of research in these areas and a wide variety of standards and guidance documents to aid in the system safety approach (see Appendix B).
Systems engineering is a way to manage complexity in design, including the potential for the developers themselves to make errors (Leveson 2016, Neumann 1994). These principles are the basis for system safety, which is a way of managing safety in a structured way. Dunn (2002) describes this as ‘a distinct set of engineering and management principles, criteria, and techniques that not only help to define safety requirements but how the design process should be structured and conducted in order to realise a safe system.’
Even for simple systems, following a structured approach like this helps the RTO and equipment developer to understand how the equipment fits within the rail system and how it can affect safety. It greatly reduces the number of ‘unknowns’. Dunn (2002) stated that safety-critical design and evaluation methods could be applied across the ‘full range of computer system applications’ from ‘the fly-by-wire airliner where large numbers of human lives are at stake down to the small manufacturing facility where human injury or product damage represent the worst-case safety outcome.’
As an example of the approach, Edwards (1997) described how a United Kingdom rail operator maintained safety assurance when commissioning a new product:
The [operator] needs to be assured that the use of a product is not going to import levels of risk that would compromise their existing safety performance. To provide this assurance the ‘sponsor’ of a product needs to demonstrate that the design, manufacture and installation was carried out in a systematic manner. Hazards were identified, risks assessed and suitable and sufficient control measures have been selected and imposed and further, that the work was carried out under a proper quality and safety management regime.
Edwards named several standards that detail assurance processes for different systems or parts of systems:
safety-related system (IEC 1508)
rail system (EN 50126)
railway signalling control and protection system that uses software (EN 50128)
safety-related railway signalling electronics (EN 50129).
Of these, EN 50126 (Railway Applications – The Specification and Demonstration of Reliability, Availability, Maintainability and Safety (RAMS)) is the most directly applicable to the safety of remotely-controlled trains. It stated that it is applicable to:
… all railway applications and at all levels of such an application, as appropriate, from complete railway systems to major systems and to individual and combined subsystems and components within these major systems, including those containing software.
There are other standards that are broadly similar, and EN 50126 is presented here as one example of the type of standard that can be applied to attain a nominal level of system safety assurance in rail. It has not yet been issued as an Australian standard, so has limited visibility within the Australian rail industry.
Another problem is that to fully understand this 670-page standard in practice requires specialist knowledge, and to apply it may require a level of human resources that are simply not available to, or too costly for, small organisations. This type of detailed and complex standard cannot be easily tailored for simple application, even though the broadest principles can still be useful.
When implemented in full, many of these system safety practices come at a significant cost—well into the millions of dollars even for relatively small projects. For this reason, it is very important to use standards that are appropriate to the application. This means the level of detail in the standard should be enough to overcome the safety needs of the application, but not so much as to make the project unviable.
In addition, standards are inherently expensive: Hobbs (2020) estimated a cost of about US$3,000 for a single-user license to read IEC 61508, adding that this may be a barrier between small organisations and legal copies of important standards.
Furthermore, someone with even a moderate level of system safety expertise might not recognise the significant differences between a claim that a product meets a standard, an assertion or certificate from a non-accredited independent body, and certification from a body that itself has been accredited to issue certification for that standard. This means that some operators might think that a system or device meets, or was developed in full accordance with, a standard when it does not. To alleviate this risk, operators can attain certification from an accredited body, or by producing or reviewing a document that sets out how each element of the standard has been met, or why it has not (for example, if an element is optional or not applicable).
At the time of 21 September 2018 runaway, the Australian standards relating to rolling stock safety were mostly what is known as ‘specification’ standards; that is, focusing on the characteristics of the end product (such as dimensions) rather than the process of design. There was no explicit regulatory requirement for developers of rail equipment to demonstrate an objective evaluation of design safety or apply system safety principles during development. An RTO could apply specification standards and not broader process standards, such as those that are used for design safety assurance.
This approach has limitations. It is simpler to prove adherence to a specification than to a process but a specification may have limited application. For example, a specification for rail track geometry cannot be applied to anything else. Specification standards also have limited effectiveness when applied to complex systems. Process standards tend to focus more on principles and process, enabling developers and operators to cope with higher levels of complexity.
There has been an increasing global emphasis on system safety in rail, and the Australian rail industry has increasingly applied systems engineering principles over the past decade, particularly in NSW and Victoria (Welschen and others 2021). However, Welshen and others also noted that the Australian transport sector lacked national direction in the application of systems engineering compared with the defence sector or that provided in other countries.
In 2013, system safety guidance material specific to the rail industry was collated into an International Engineering Safety Management Handbook,[90] which was aimed at ‘clearly outlining the activities involved in making a system or product safe and providing the evidence that it is safe.’ The extent to which this and other guidance reached the Australian rail industry is unclear. At the time of the runaway, there was no equivalent guidance with high visibility and relevance to the Australian rail industry. In addition to the runaway involving the TasRail cement train no. 604, the runaway involving the BHP iron ore train in November 2018 also involved problems with the integration of complex systems associated with the non-application of a systems engineering framework.[91]
Of the standards that were highly visible and applicable to the Australian rail industry, AS 7501 (Rolling stock compliance certification) was relatively simple to apply but, in addition to being optional, did not directly lead RTOs to develop a safety case or perform safety analyses. This was done in AS 7702 (Rail equipment type approval), but this standard primarily did not apply to rolling stock. Similarly, ONRSR’s guidance on major projects listed several activities that could be beneficial for smaller projects that are still complex and safety-critical, such as the development of the generation 3 RCE.
By contrast, the manufacture and modification of transport category aircraft (which includes almost all commercial passenger or cargo aircraft) has been governed by regulations requiring equipment, systems, and installations to be designed to reduce the likelihood of unsafe failure conditions to a specified amount.[92] Compliance must be shown by analysis and, where necessary, tests that consider:
possible modes of failure including malfunctions and damage from external sources
the probability of multiple failures and undetected failures
the resulting effects in different operating phases and conditions, and
warning cues, corrective actions, and fault-detection capability.
Where these rules applied, there was generally associated guidance to show acceptable means of compliance. For example, in the United States, Advisory Circular 25.1309 provided extensive details about technical considerations that should be made and lists several aviation system safety standards that can be used to show compliance. [93]
In conclusion, although standards, legislative requirements, and guidance for rail safety were recognised and applied in Australia, they were of limited value in the development and operation of complex systems. Conversely, the standards and guidance for system safety available at the time of the runaway were either:
too abstract, complex, costly and/or impractical for widespread recognition and acceptance by the Australian rail industry
not required to be applied to rolling stock (AS 7702), or
not applicable to projects that were complex but not ‘major’ (ONRSR Guideline – Major Projects).
If an RTO or equipment developer intended to apply system safety principles, there were still numerous challenges in attaining justified confidence in its success, which were difficult even for experts to overcome. Finally, under Australia’s co-regulatory rail safety framework, there was no explicit regulatory requirement to apply system safety processes in the Australian rail industry, and minimal guidance other than for ‘major’ projects.
Findings
ATSB investigation report findings focus on safety factors (that is, events and conditions that increase risk). Safety factors include ‘contributing factors’ and ‘other factors that increased risk’ (that is, factors that did not meet the definition of a contributing factor for this occurrence but were still considered important to include in the report for the purpose of increasing awareness and enhancing safety). In addition ‘other findings’ may be included to provide important information about topics other than safety factors.
Safety issues are highlighted in bold to emphasise their importance.A safety issue is a safety factor that (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.
These findings should not be read as apportioning blame or liability to any particular organisation or individual.
From the evidence available, the following findings are made with respect to the runaway and derailment of TasRail freight train in Devonport, Tasmania on 21 September 2018.
Contributing factors
While the driver was conducting loading operations at Railton (from outside of the train), the remote control equipment (RCE) entered a spurious fault condition (involving the application of emergency braking) that was almost certainly associated with the RCE’s response to rapid movement of the RCE transmitter’s direction controller. This likely led to the RCE entering a persistent unresponsive state in which the train’s brakes were not applied, allowing the train to roll away.
The TasRail cement loading facility at Railton had a downhill grade to the main line, and no devices to protect against a runaway. (Safety issue)
After the train rolled away from Railton, the remote control equipment’s receiver did not apply the train’s brakes as designed when outside radio communication range. The train’s brakes remained in a released (unapplied) state until the derailment.
The Air Digital Engineering generation 3 remote control equipment (RCE) had several safety-related design and integration problems, which were readily identifiable. These included:
unintended activation-and-release of emergency braking on the locomotive
recovery from an emergency brake application and certain penalty states that was inconsistent with locomotive braking system timeout controls
the potential to enter a persistent unsafe state during initialisation, which was unrecoverable without external intervention
the absence of a means to detect and respond to an emergency brake application from a source external to the RCE
the vigilance and driver-commanded emergency stop functions being unavailable in the absence of an active radio communications link. (Safety issue)
Although Air Digital Engineering had safety as a design objective and safety elements were included in the remote control equipment, system safety assurance activities appropriate to its application were not conducted. (Safety issue)
TasRail commissioned the manufacture of, and continued to use, redesigned safety-critical remote control equipment for operating a locomotive without systematic assurance of its safety, leading to excessive reliance on the manufacturer. This was because TasRail did not:
fully engage with the development process from initial design through to commissioning, or understand the extent of design changes associated with the introduction of the generation 3 RCE (which reduced the capability to prevent, identify and resolve design and integration problems)
explicitly identify and impose safety requirements
verify that the overall system met a specified level of safety. (Safety issue)
Although there were no previous accidents attributable to TasRail’s use of remote control equipment (RCE) over 19 years, TasRail did not identify or fully assess the safety implications of remotely-controlled train operations, or those of TasRail’s specific implementation. These included the:
potential for remote control equipment to fail to an unsafe state without external risk controls
suppression or absence of some safety functions of the TR class locomotive, including overspeed protection and some indications provided to drivers
non-recording of key data parameters to facilitate effective fault and incident analysis
increased risk of derailment as a result of motive power located only at the rear of the train. (Safety issue)
Associated with the limited internal communication about the extensive changes from the second to the third generation remote control equipment, TasRail did not submit a notification of change to the Office of the National Rail Safety Regulator. As a result, regulatory oversight of the project was not initiated.
Although TasRail had a detailed change management process in place, and had documented that the project to develop the third-generation remote control equipment was a significant change, the change management process had a limited capability to:
assure pre-determined activities, approvals, and documentation were completed throughout progression of a change
identify the need for relevant safety assurance activities
assure the determination of whether a change had the potential to impact safety. (Safety issue)
TasRail did not have a reliable process to systematically identify, track and analyse reported faults on its remotely-controlled train or to identify their potential safety implications. (Safety issue)
There was limited practical guidance specifically for the Australian rail industry for the application of system safety assurance processes to the development of complex and safety-critical rail systems. (Safety issue)
Other factors that increased risk
Although TasRail had documented that the project to develop the generation 3 remote control equipment was a significant change, most pre-determined activities required to be conducted for that type of change were not performed during the project.
The guidance provided by the Office of the National Rail Safety Regulator about the requirement to submit a notification of change included limited detail about the extent or type of changes that necessitated a notification. In addition, with regard to ‘a safety critical element of rolling stock’, it did not provide detail with regard to the interpretation of ‘safety critical’ and the applicability to equipment that may not be inherently part of rolling stock (such as remote control equipment). (Safety issue)
TasRail’s processes for ensuring immediate network control actions in response to emergencies (such as runaway and authority exceedance) fundamentally relied on the experience and knowledge of network control officers and did not include the provision of procedures, tools and checklistsdetailed enough to support the effective management of specific types of incidents that require a time-critical response. (Safety issue)
Although the Office of the National Rail Safety Regulator required TasRail to have an operating event recorder fitted to its trains, this requirement did not necessitate that sufficient parameters of the remote control equipment (RCE) were recorded for optimal outcomes of internal and external safety investigations into events involving use of the RCE.
Other findings
The driver, TasRail network control and other staff, and emergency services conducted a prompt and effective response to minimise consequence as the runaway train descended into Devonport.
Safety issues and actions
Central to the ATSB’s investigation of transport safety matters is the early identification of safety issues. The ATSB expects relevant organisations will address all safety issues an investigation identifies.
Depending on the level of risk of a safety issue, the extent of corrective action taken by the relevant organisation(s), or the desirability of directing a broad safety message to the rail industry, the ATSB may issue a formal safety recommendation or safety advisory notice as part of the final report.
All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.
Descriptions of each safety issue, and any associated safety recommendations, are detailed below. Click the link to read the full safety issue description, including the issue status and any safety action/s taken. Safety issues and actions are updated on this website when safety issue owners provide further information concerning the implementation of safety action.
Safety issue description: The TasRail cement loading facility at Railton had a downhill grade to the main line, and no devices to protect against a runaway.
Remote control equipment design and integration problems
Safety issue description: The Air Digital Engineering generation 3 remote control equipment (RCE) had several safety-related design and integration problems, which were readily identifiable. These included:
unintended activation-and-release of emergency braking on the locomotive
recovery from an emergency brake application and certain penalty states that was inconsistent with locomotive braking system timeout controls
the potential to enter a persistent unsafe state during initialisation, which was unrecoverable without external intervention
the absence of a means to detect and respond to an emergency brake application from a source external to the RCE
the vigilance and driver-commanded emergency stop functions being unavailable in the absence of an active radio communications link.
Safety issue description: Although Air Digital Engineering had safety as a design objective and safety elements were included in the remote control equipment, system safety assurance activities appropriate to its application were not conducted.
Safety issue description: TasRail commissioned the manufacture of, and continued to use, redesigned safety-critical remote control equipment for operating a locomotive without systematic assurance of its safety, leading to excessive reliance on the manufacturer. This was because TasRail did not:
fully engage with the development process from initial design through to commissioning, or understand the extent of design changes associated with the introduction of the generation 3 RCE (which reduced the capability to prevent, identify and resolve design and integration problems)
explicitly identify and impose safety requirements
verify that the overall system met a specified level of safety.
Safety issue description: Although there were no previous accidents attributable to TasRail’s use of remote control equipment (RCE) over 19 years, TasRail did not identify or fully assess the safety implications of remotely-controlled train operations, or those of TasRail’s specific implementation. These included the:
potential for remote control equipment to fail to an unsafe state without external risk controls
suppression or absence of some safety functions of the TR class locomotive, including overspeed protection and some indications provided to drivers
non-recording of key data parameters to facilitate effective fault and incident analysis
increased risk of derailment as a result of motive power located only at the rear of the train.
Safety issue description: Although TasRail had a detailed change management process in place, and had documented that the project to develop the third-generation remote control equipment was a significant change, the change management process had a limited capability to:
assure pre-determined activities, approvals, and documentation were completed throughout progression of a change
identify the need for relevant safety assurance activities
assure the determination of whether a change had the potential to impact safety.
Safety issue description: TasRail did not have a reliable process to systematically identify, track and analyse reported faults on its remotely-controlled train or to identify their potential safety implications.
System safety assurance guidance for the Australian rail industry
Safety issue description: There was limited practical guidance specifically for the Australian rail industry for the application of system safety assurance processes to the development of complex and safety-critical rail systems.
Notification of change regulatory requirements and guidance
Safety issue description: The guidance provided by the Office of the National Rail Safety Regulator about the requirement to submit a notification of change included limited detail about the extent or type of changes that necessitated a notification. In addition, with regard to ‘a safety critical element of rolling stock’, it did not provide detail with regard to the interpretation of ‘safety critical’ and the applicability to equipment that may not be inherently part of rolling stock (such as remote control equipment).
Safety issue description: TasRail’s processes for ensuring immediate network control actions in response to emergencies (such as runaway and authority exceedance) fundamentally relied on the experience and knowledge of network control officers and did not include the provision of procedures, tools and checklists detailed enough to support the effective management of specific types of incidents that require a time-critical response.
Safety action not associated with an identified safety issue
Whether or not the ATSB identifies safety issues in the course of an investigation, relevant organisations may proactively initiate safety action in order to reduce their safety risk. The ATSB has been advised of the following proactive safety action in response to this occurrence.
Additional safety action taken by TasRail
Shortly after the accident, TasRail indicated that it would be reducing the overspeed limit on the TR class locomotive from 88 km/h to 75 km/h. This reflected the maximum track speed on the TasRail network of 70 km/h.
In September 2019 TasRail presented its internal findings and lessons to the rail industry in a forum co-ordinated by the Rail Industry Safety Standards Board (RISSB) titled ‘Lessons Learned from Investigations’.
Glossary
ACMA Australian Communications and Media Authority
ADE Air Digital Engineering
ANCS Advanced Network Train Control System. The safeworking system used on the TasRail network, which used GPS tracked electronic track warrants and authorities.
AS Australian Standard
CHU Cab handle unit. Manipulated by the driver, this sent electronic airbrake requests to the POU to enact.
COMF mode Communication failed mode. Fault mode of the RCE in which traction power was removed and the train’s brakes applied after an interruption of communications between the RCE transmitter and RCE receiver.
CNF mode Control fault mode. Fault mode of the RCE in which traction power was removed and the train’s brakes applied where the RCE receiver or locomotive response did not replicate the driver’s RCE transmitter command.
Driver’s van A reclaimed Y-class locomotive with no engine or traction motors, used for the driver to observe the track ahead while propelling the consist with the remotely-controlled locomotive at the rear of the train.
EMV Emergency magnet valve. Assists in localised rapid brake pipe exhaust in the event an emergency rate reduction of the brake pipe was detected.
Emergency brake The application of maximum braking effort throughout the train by venting the automatic brake pipe, or in the case of brake pipe rupture.
Emergency stop A function provided by the RCE to apply emergency braking and command the removal of traction power, among other actions. Emergency stop mode (a software mode) could be driver-initiated, or RCE-initiated (in the event of a fault). Some RCE hardware faults would result in the application of emergency brake only, without the RCE entering emergency stop mode.
Fail-safe The capability (or design approach intended to provide the capability) of a system such that any failure always results in a safe condition.
FMEA Failure modes and effects analysis. A systematic approach to document the potential effects of component failures in a system.
LCD Liquid crystal display. A type of flat panel display.
MR1 Main reservoir no. 1. The locomotives primary store of pressurised air. Fed by the locomotive air compressor and supplying air to all subsequent systems, including main reservoir no. 2 and the main reservoir equalising pipe.
MR2 Main reservoir no.2. A secondary source of air pressure on the locomotive fed by main reservoir no. 1 through a one-way valve, this air was used on a non-remotely-controlled train to recharge the brake pipe, charge the control pipe and brake cylinders.
MREQ Main reservoir equalising pipe. This allowed a continuous supply of air from main reservoir no. 1. Its intend function was to achieve consistent pressure in the main reservoir no. 1 throughout all locomotives on a train. On an RCE equipped train, this air was also used to recharge the brake pipe and charge the control pipe.
ONRSR Office of the National Rail Safety Regulator
PCS Power control switch. Automatically removed traction power and dynamic brake from the locomotive when low brake pipe pressure was detected.
POU Pneumatic operating unit. Located in the locomotive’s engine room, it enacted the air pressure responses for the train’s electronic airbrake system.
RCE Remote control equipment. This consisted of a transmitter (for driver commands), a receiver box, and air box as well as associated multiple unit cable, airbrake hoses and radio antennas.
RISSB Rail Industry Safety and Standards Board
RSNL Rail Safety National Law
RTO Rail transport operator. Encompassed both rail infrastructure managers (track, signalling etc.) and rolling stock operators (locomotives, wagons etc.).
Safe state A state or mode that a system can enter, usually following a failure, which is associated with an acceptable level of risk.
SAL System for automated locomotive computer control. A proprietary system in the TR class locomotive that included the capability to monitor and control locomotive traction power, record and indicate faults, and allow diagnostic testing.
Selcall Selective calling on a radio system, whereby receiving systems would selectively answer a radio transmission based on the sequential audio tone transmitted.
SIL Safety integrity level. A number from 1 (the lowest) to 4 (the highest) which specified the probability of a system meeting its safety functions under all stated conditions within a stated period of time.
SMS Safety management system. A systematic approach to organisational safety encompassing safety policy and objectives, risk management, safety assurance, safety promotion, third party interfaces, internal investigation and SMS implementation.
VX vent valve A valve designed to locally vent the brake pipe on detecting a rapid rate (emergency) of brake pipe descent. Two were fitted to the TR class locomotive.
Sources and submissions
Sources of information
The sources of information during the investigation included:
TasRail (rolling stock operator and rail infrastructure manager)
Air Digital Engineering (RCE manufacturer)
Progress Rail Services (TR class locomotive manufacturer)
Wabtec Australia (electronic airbrake system and wagon braking system manufacturer)
the Office of the National Rail Safety Regulator
Tasmanian Government (Department of State Growth)
Tasmania Police
the driver of train no. 604
network control centre personnel
members of the generation 3 RCE project team
accident witnesses
front-of-train closed-circuit television
recorded data from the locomotive’s 2 recording devices
rail industry standards, guidelines, codes of practice
engineering and system safety standards
legislative instruments
data and observations from extensive post-accident testing of the train and RCE.
References
Dismukes RK & Berman BA 2010, Checklists and monitoring in the cockpit: why crucial defences sometimes fail, National Aeronautics and Space Administration Technical Memorandum NASA/TM-2010-216396.
Dunn WR 2002, Practical Design of Safety-Critical Computer Systems. Reliability Press, Solvang.
Ericson CA 2005, Hazard analysis techniques for system safety. John Wiley & Sons, New Jersey.
Federal Railroad Administration (FRA) 2005, Safety of Remote Control Locomotive (RCL) Operations. Report to Congress, Washington.
Federal Railroad Administration (FRA) 2006a. A Comparative Risk Assessment of Remote Control Locomotive Operations versus Conventional Yard Switching Operations. Technical Report DOT/FRA/ORD-06/09, Washington.
Federal Railroad Administration (FRA) 2006b. Human Factors Root Cause Analysis of Accidents/Incidents Involving Remote Control Locomotive Operations. Technical report DOT/FRA/ORD-06/05, Washington.
Federal Railroad Administration (FRA) 2006c. Remote Control Locomotive Operations: Results of Focus Groups with Remote Control Operators in the United States and Canada. Technical report DOT/FRA/ORD-06/09, Washington.
Hawkins, R, Habli I and Kelly T 2013. The Principles of Software Safety Assurance. In 31st International System Safety Conference, Boston.
Hawkins RD and Kelly TP 2009. Software Safety Assurance – What Is Sufficient?. In 4th IET International Conference on Systems Safety, London.
Hawkins RD and Kelly TP 2012. A framework for determining the sufficiency of software safety assurance. In 7th IET International Conference on System Safety, Edinburgh.
Hobbs C 2020. Embedded Software Development for Safety-Critical Systems (2nd ed.). CRC Press, Boca Raton.
Kusumo R (2019) A systems approach to safe system integration in major rail projects, paper presented at the Rail Industry Safety and Standards Board (RiSSB) Rail Safety Conference in Melbourne, May 2019.
Leveson NG 2016. Engineering a Safer World: Systems Thinking Applied to Safety. The MIT Press, Cambridge, Massachusetts.
Littlewood B and Strigini L 1995. Validation of Ultra-High Dependability for Software-Based Systems. In Predictably Dependable Computing Systems (pp. 473-493). Springer, Berlin, Heidelberg.
McInerney, P 2001. Special Commission of Inquiry Into the Glenbrook Rail Accident – Final Report.
McInerney, P 2005. Special commission of inquiry into the Waterfall rail accident – Final Report.
Ministry of Defence (United Kingdom) 2007. Safety Management Requirements for Defence Systems, Defence Standard 00-56.
National Research Council 2007. Software for Dependable Systems: Sufficient Evidence?. The National Academies Press, Washington.
Neumann PG 1994, Computer-related risks. Addison-Wesley, New York.
Rail Industry Safety and Standards Board (RISSB) 2014, AS7510.1: Braking systems part 1 – locomotive rolling stock, RISSB, Spring Hill Queensland Australia
Rail Industry Safety and Standards Board (RISSB) 2015, AS7527 – Rolling stock event recorders, RISSB, Spring Hill Queensland Australia
Rail Industry Safety and Standards Board (RISSB) 2015 (2019 amendment), AS7527 – Rolling stock event recorders, RISSB, Spring Hill Queensland AustraliaRail Industry Safety and Standards Board (RISSB) 2018, Code of practice: Distributed power freight trains, RISSB, Spring Hill Queensland Australia
Smith DJ and Simpson KG 2010. Safety critical systems handbook: a straight forward guide to functional safety, IEC 61508 (2010 Edition) and related standards, including process IEC 61511 and machinery IEC 62061 and ISO 13849. Elsevier.
Welschen R, Bellon E, Brown C, Fullalove R, Kennedy G, Irvine K, Mumford N, Nadeem M, Nasr J, Tildesley E, Patel H, and Roodt D (2021), An Overview of Systems Engineering in the Australian Transport Sector. Systems Engineering News Journal, March.
Submissions
Under section 26 of the Transport Safety Investigation Act 2003, the ATSB may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. That section allows a person receiving a draft report to make submissions to the ATSB about the draft report.
A draft of this report was provided to the following directly involved parties:
Air Digital Engineering (ADE)
TasRail
the Office of the National Rail Safety Regulator (ONRSR)
the Rail Industry Safety and Standards Board (RISSB)
the driver of train no. 604
Wabtec Australia (the train braking system manufacturer).
Submissions were received from:
Air Digital Engineering (ADE)
TasRail
the Office of the National Rail Safety Regulator (ONRSR).
The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.
Appendices
Appendix A – Analysis of occurrence event information
Examinations at the accident site
On 25 September 2018, the remote control equipment (RCE) attached to the TR class locomotive TR11 was removed in the presence of the ATSB for removal for storage in a secure location. Prior to removal, a visual inspection was undertaken, with no obvious faults or problems detected.
Following the visual inspection, the air hoses connecting the TR class locomotive to the RCE equipment and aerial connections were progressively removed and insulation resistance tested. This included a standing wave ratio test of the RCE antenna and coaxial cable. This testing also did not identify any obvious faults or problems.
Overview of subsequent examination and testing
During 2 periods in October 2018, preliminary testing of the RCE involved in the accident with both locomotive TR11 and another TR class locomotive was undertaken. This included testing both with the ATSB in attendance and on its behalf. Tests included electronic airbrake transducer calibration and self-test diagnostics. Preliminary RCE fault response testing was also undertaken.
In late October 2018, further locomotive and RCE testing was undertaken by a consultancy firm that was engaged by TasRail to provide an internal report into the accident. The consultancy firm also undertook extensive testing of radio coverage and interference between Devonport and Railton and assessments of cyber security risk.
There was no data recorded that included RCE modes, driver control positions on the RCE transmitter, commands from the transmitter to the receiver, or commands from the receiver to the air box or locomotive. As a result, the ATSB analysis was based on other data recorded on the locomotive throughout the runaway sequence, as well as the driver’s recollection.
Available documentary evidence related to the design and operation of the generation 3 RCE was reviewed, including the various modes, configurations and responses described with the RCE operation manual and its interface with the TR class locomotive and electronic airbrake system. The ATSB also carried out extensive interviews with TasRail, Air Digital Engineering (ADE), and the locomotive and braking system manufacturers.
However, the available data was insufficient to completely determine the runaway sequence and important aspects of RCE behaviour so the ATSB determined that further RCE testing was required.
Accordingly, additional testing was undertaken by the ATSB over a 2-day period in February 2020 with the assistance of TasRail. This testing was carried out with the RCE involved in the accident, another TR class locomotive, and a similar train configuration to that of the cement train. TasRail representatives and 2 rail safety officers from the Office of the National Rail Safety Regulator (ONRSR) also attended these tests.
The focus of these tests was software and hardware fault responses and RCE–locomotive–train integration. Part of these tests included fault and integration responses of the TR class locomotive. The testing included multiple onsite tests of the generation 3 RCE, in conjunction with different train consists, to gather information relating to possible failure scenarios and locomotive responses. Simulations reproducing some characteristics of the occurrence sequence were also undertaken, using available driver commands to replicate observed RCE and train behaviour. The testing was facilitated by both TasRail and ADE.
In addition to on-train tests, the RCE was tested separately to the locomotives and train consists in both October 2018 and February 2020, on the test bench at the RCE maintenance contractor’s premises. Part of these tests included software and hardware behaviour, physical inspection of the equipment (including air valves), and a test of the transmitter’s battery.
ADE later reported:
ADE has conducted a hardware & software analysis of its factory test system which is software, electronically and electrically identical to the RCE Gen 3 system on the locomotive at the time of the accident. ADE could not reproduce a condition that led to the events preceding the accident with regard to Brake Pipe changes ... ADE was primarily interested in trying to create and or identify the catalyst which set the [accident] sequence of events in motion [with regard to the rapid reverser movement[94] and potential persistent unsafe state during initialisation[95]].
The testing was limited to putting the software and hardware under stress in as much as possible within the limitation of the equipment not being connected to a locomotive. This involved for periods of time (10 minutes) manually and consistently rapidly transmitting control commands putting the system under executional stress. It also included the consistent rapid changing of the reverser...
The testing concluded that:
the locomotive, including its control and safety systems, operated as designed
the locomotive electronic airbrake system operated as designed
there was no evidence that cyber intrusion occurred.
Analysis of the recorded data for the runaway and derailment is presented below in Table A1, Table A2 and Table A3, with relevant parameters combined from the available data log’s presented in Figure A1, Figure A2 and Figure A3.
Remote control equipment failure
Table A1: Sequence during remote control equipment failure (0841:55 to 0843:55)
Time
Observation
Analysis
0841:55 to 0842:29
Label A1: brake pipe pressure increased
Label A2: direction controller state changed from neutral to forward
Responses are consistent with the driver’s description that, after completing loading wagons 13 and 14, they moved the train forward to position the last 2 wagons for loading. Automatic and independent brake were released, direction selected, and power applied causing the train to move.
0842:32
Label A6: brake pipe pressure decreased
Label A7: locomotive brake cylinder pressure increased momentarily then decreased
Responses are consistent with an automatic (train) brake application, where the locomotive brakes have been ‘bailed-off’ to prevent them from applying.
Although an automatic train brake application was not described by the driver, the use of automatic brake in conjunction with ‘bail-off’ in this manner is normal to control movement at low speed. Therefore, it is likely that this was requested by the driver.
0842:52
Label A8: locomotive brake cylinder pressure increased, corresponding to a further decrease in brake pipe pressure
The observed behaviour is consistent with further application of the automatic train brake, without a corresponding bail-off command, resulting in the locomotive’s brakes applying.
The driver observed that the train would, and subsequently did, overshoot the intended stopping point. The driver did not describe requesting an increased brake application. However, increased braking effort is a consistent response when stopping a train.
0842:55
Label A9: throttle reduced to idle
Label A10: brake pipe pressure increases
The observed behaviour is consistent with the driver responding to the overshoot and preparing to reverse the train by removing the throttle and releasing the automatic brake.
Note: The observations and analysis (labels A1 to A10) above indicate that the train was responding to remote control commands, as the driver requested, during this time.
0842:57
Label A11: direction controller state changed from forward to reverse
The driver recalled requesting the direction to change in preparation to reverse the train. The data was recorded at 1-second intervals and the direction controller state went immediately from forward to reverse without registering in neutral. This indicated that the change of direction request occurred in under 1 second, likely because the driver quickly selected the reverse direction in a straight sweep from forward, without pausing in neutral.
This is the last observation consistent with the driver’s requested commands.
0842:57
Label A12: locomotive brake cylinder pressure increased to 337 kPa
The driver advised that, in conjunction with the change in direction (label A11) they requested an independent brake application. Although the driver did not specify what level of independent brake application they requested, observation of previous applications during loading / unloading operations showed that they almost always resulted in about 500 kPa in the locomotive brake cylinders.
The low brake cylinder pressure achieved on this occasion, not exceeding 337 kPa, indicated that it was likely that the RCE did not charge the control pipe in response to the driver’s request. The locomotive brake cylinder pressure was the result of low brake pipe pressure only, consistent with an automatic brake application.
0842:57
Label A13: the throttle remained at idle when the direction controller state changed from forward to reverse (label K)
The driver advised that, in conjunction with change of direction (label A11), they requested a throttle application. However, the recorded data does not show a corresponding locomotive response, and the throttle continued to remain at idle through the rest of the sequence.
0842:57
Label A14: at the same time that the direction controller state recorded reverse (label A11), the brake pipe pressure decreased rapidly to about 100 kPa
The brake pipe pressure rapidly decreased to about 100 kPa, consistent with an emergency brake application. It is highly likely that this was the result of a dual direction fault interlock as it occurred in conjunction with the rapid change of direction, which is a known trigger for this response. When this state was detected, the RCE receiver responded by commanding a vent of the brake pipe through its emergency valve, initiating the rapid pressure decrease. The RCE emergency vent valve only remained open momentarily while the fault state was present.
The locomotive’s VX vent valves and emergency vent valve (EMV) likely responded to the momentary rapid pressure decrease and activated, venting the brake pipe at additional locations separate to the RCE.
0842:58 to 0843:26
Label A15: main reservoir no. 1 pressure decreased while main reservoir no. 2 pressure remained relatively constant
The loss of air pressure from main reservoir no. 1, without corelated loss from main reservoir no. 2, was consistent with an attempt by the RCE to charge the brake pipe.[96] Therefore it is likely that, once the RCE dual direction fault interlock cleared, it attempted to increase the brake pipe pressure in opposition to the locomotive’s VX vent valves and EMV, which were both venting the brake pipe air. This resulted in the brake pipe pressure being maintained well below 350 kPa, but above zero (the expected result of an emergency vent).
0843:03
Label A16: brake pipe pressure increased, reaching a maximum of about 220 kPa before slowly decreasing again to about 180 kPa
The increase likely corresponded to the VX vent valves ceasing to vent brake pipe air. (The VX vent valves are self-regulating and cease to vent when a volume of air, called the control pressure volume, has vented.)
0843:18
Label A17: brake pipe pressure again increased
The second increase likely corresponded with the locomotive’s EMV closing, consistent with its 20-second timeout after initial activation in response to a rapid brake pipe pressure loss. This allowed the RCE to recharge the brake pipe without opposition. The maximum brake pipe pressure reached was 468 kPa.
0843:24
Label A18: locomotive brake cylinder pressure decreased as brake pipe pressure rose above about 320 kPa
The brake cylinder pressure did not begin to reduce in response to the initial rise in brake pipe pressure. Instead, it is likely that the brake cylinder pressure release was delayed until brake cylinder and brake pipe pressures had equalised, an action of the locomotive’s electronic airbrake system.
Note: The ATSB observed the behaviour described (labels A14 to A18) to occur repeatably during testing conducted after the runaway. This was triggered when the driver rapidly made a direction change request on the RCE transmitter.
Figure A1: Combined data logger data during remote control equipment failure (0841:00 to 0844:30)
Source: ATSB
Initial runaway
Table A2: Sequence of the initial runaway (0843:55 to 0846:42)
Time
Observation
Analysis
0843:55
Label B1: brake pipe pressure decreased rapidly to about 0 kPa in both Figure A1 and Figure A2
Label B2: main reservoir no. 1 pressure continued to increase
Label B3: direction controller state remained in reverse
The brake pipe pressure rapidly decreased to about 0 kPa, similar to an emergency brake application.
The locomotive’s VX vent valves and EMV likely responded to the momentary rapid pressure decrease and activated, venting the brake pipe at additional locations separate to the RCE.
The main reservoir no. 1 pressure did not decrease in response to this emergency vent. This indicated that, unlike the prior vent, the RCE did not attempt to recharge the brake pipe while the locomotive valves were open. Therefore, it is likely that the RCE initiated and maintained the emergency brake application.
The direction controller state was maintained in ‘reverse’. This behaviour was not consistent with the RCE’s emergency mode or any other documented RCE fault condition.
0844:28
Label B4: brake pipe pressure increased rapidly from 0 kPa to above 480 kPa
The rapid rise of brake pipe pressure observed was consistent with the RCE air box supplying air to the brake pipe at the rear of the locomotive. Because the location of the pressure sensor was in front of the air supply, it was not a true measure of the brake pipe pressure throughout the train and instead increased more rapidly than was normal for a train of 220 m length.
No reason could be determined for the RCE to initiate a recharge of the brake pipe at this time. The recharge resulted in a maximum brake pipe pressure of 489 kPa and no pressure in the locomotive brake cylinders. The rise in brake pipe pressure would have provided a release command to the brakes on the wagons (an automatic brake release command).
0844:32
Label B5: locomotive brake cylinder pressure decreased as brake pipe pressure rose above about 320 kPa
The brake cylinder pressure did not begin to reduce in response to the initial rise in brake pipe pressure. Instead, it is likely that the brake cylinder pressure release was delayed until brake cylinder and brake pipe pressures had equalised, an action of the locomotives electronic airbrake system.
0844:53
to 0845:33
Label B6: brake pipe pressure decreased rapidly to about 0 kPa
Label B7: main reservoir no. 1 pressure continued to increase
Label B8: direction controller state remained in reverse
Label B10: locomotive brake cylinder pressure decreased as brake pipe pressure rose above about 320 kPa
The sequence of the third emergency brake application repeated the second emergency brake application (labels B1 to B5 above).
ATSB analysis indicated:
The behaviour was not consistent with the RCE’s emergency mode or any other documented RCE fault condition.
No reason could be determined for the RCE to initiate a recharge of the brake pipe. The brake pipe pressure recharged to 489 kPa.
The train was left with no brake applications commanded once the brake pipe pressure recharged, though effective release of the wagon brakes would have taken some time.
Note: The condition observed at the end of the second and third emergency brake applications, with the brake pipe pressure at or near the release level, no brake cylinder pressure and with the direction controller state not in ‘centre’, was not consistent with any RCE penalty mode or the configuration of the controller required to recover from a penalty mode (the ‘start position’[97]).
The driver reported that, during this time, they:
believed that the RCE was in a communication failed mode
had configured the transmitter in the start position
attempted to reset the transmitter to re-establish a communication link with the receiver.
The recorded information does not show evidence that the communication failed mode or the driver’s commands were actioned by the RCE receiver or enacted by the locomotive.
0846:42
Label C1: the train’s speed began to increase, observable in both Figure A2 and Figure A3
With the brakes released, the train began to roll on the downhill grade toward Devonport.
Note: The ATSB conducted testing on a similar train consist to the cement train to examine the behaviour of the wagon brakes in response to the brake pipe behaviour recorded at the locomotive. The results of these tests showed that:
After the third emergency brake application the wagon brake system had sufficient air remaining to apply if commanded (that is, the train’s brake system had not run out of air). This was determined by observation of the auxiliary reservoir pressure in wagons throughout the train, which showed sufficient pressure remained (wagons 1, 8 and 16 did not fall below 300 kPa, 280 kPa and 275 kPa, respectively) to apply the wagon brake cylinders to an effective level.
Wagon brake cylinders similar to those fitted to the cement wagons took over 80 seconds to fully release from about 300 kPa. Wagons to the rear of the train (being farther from the source of the brake pipe recharge) took longer than the front wagons to release. This was consistent with the delay of 72 seconds observed from the final release command to the train beginning to roll away.
Figure A2: Combined data logger data during initial runaway (0843:30 to 0847:00)
Source: ATSB
Continued runaway and derailment
Table A3: Sequence of continued runaway and derailment (0846:42 to 0914:24)
Time
Observation
Analysis
The following observations (labels C1 to C7) occurred while the train was in motion during the runaway.
–
Label C1: main reservoir no. 1 pressure decreased and then increased coincident with a gradual increase in brake pipe pressure from 489 kPa to 501 kPa
These observations together were consistent with the cement train wagon auxiliary reservoirs continuing to recharge from the brake pipe. Similar main reservoir no. 1 pressure behaviour was observed to occur after the third emergency brake cycle during ATSB simulations of the sequence and did not relate to any apply or release of train brakes.
0846:42 to 0909:29
Label C2: brake pipe pressure remained at the release level
Label C3: no locomotive brake cylinder pressure was applied
Label C4: direction controller state remained in reverse
The observations are not consistent with the expected configuration in the communication failed mode. Therefore, although the driver reported observing that the transmitter was not linked to the receiver, and the receiver (on the train) exited radio range with the transmitter (being held by the driver at Railton), this did not result in the RCE entering the communication failed mode. The reason that the RCE failed to enter this mode could not be determined.
0846:42 to 0909:29
Label C5: The electronic airbrake status (EAB status) remained cut-out (C/O) throughout the event sequence
Label C6: main reservoir no. 1 pressure remained relatively constant with no large decreases
The configuration of the locomotive braking system (trail cut-out) and the behaviour of main reservoir no.1 indicated that the RCE receiver remained in control of the train’s brake systems during the runaway. During this time, the RCE did not command brake application and there was no indication that it prevented any external brake application (for example, by the locomotive).
That the locomotive remained in the ‘trail cut-out’ configuration (being controlled by the RCE) was supported by other parameters not being active; the equalising reservoir pressure and brake pipe charging flow (not shown). These parameters are only active when the locomotive is configured in ‘lead cut-in’, which was not the case.
0846:42 to 0909:29
Label C7: the throttle position remained in idle and no traction motor power (load) was recorded
The locomotive was not under traction power during the runaway and moved solely under the force of gravity on the downhill grade to Devonport.
The sudden stop occurred when the train derailed at Devonport. The brake pipe and brake cylinder behaviour were consistent with damage caused by the derailment. The brake pipe was likely damaged, causing it to rapidly leak, and the brake cylinders were likely also damaged as they initially applied but were unable to retain pressure.
The locomotive’s VX vent valves and EMV would have responded to the rapid leak of brake pipe pressure and activated, venting the brake pipe at additional locations separate to both the leak and the RCE.
0909:49
Label C11: the brake pipe pressure remained above 0 kPa and increased to about 68 kPa after the derailment
The brake pipe pressure did not reach 0 kPa even though it vented. Instead, it increased to 68 kPa after about 20 seconds, consistent with the EMV closing after timeout (the VX vent valves having already done so). This created more obstruction to brake pipe venting. That the pressure rose at this time indicated that the brake pipe was still being supplied with pressurised air.
While it is more likely than not that the RCE was supplying air from main reservoir no. 1 in an attempt to recharge the brake pipe to the desired level, this was not visible within the recorded data as, unlike on previous occasions, both main reservoir no. 1 and main reservoir no. 2 air pressure were decreasing.
0914:24
Label C12: locomotive engine RPM decreases to zero
The locomotive’s engine stopped, corresponding to an emergency shutdown of the TR class locomotive by emergency services at this time. This in turn caused the locomotive’s compressor to stop, and remaining air pressure could no longer be maintained.
Figure A3: Combined data logger data during continued runaway and derailment (0838 to 0918)
Based on the available evidence, the ATSB observed that the following occurred during the occurrence sequence:
The RCE became unresponsive to driver commands at the point of, or soon after, a fast direction change from forward to reverse.
The RCE initiated a momentary emergency vent of the brake pipe, likely due to a momentary dual direction fault being detected. The emergency vent resulted in the locomotive’s VX valve and EMV activating to exhaust the brake pipe further, which the RCE tried to oppose after the fault state cleared by supplying pressurised air from main reservoir no. 1 to the brake pipe.
The RCE triggered 2 further emergency brake applications and releases.
The RCE ultimately entered an unsafe state with the brakes on the train and locomotive fully released, allowing the train to roll away on the downhill grade towards Devonport.
The RCE did not command a communication failed mode and apply the train’s automatic brake, either when either the transmitter and receiver were unlinked, or once the receiver was out of radio range of the transmitter.
The emergency brake applications and releases, and subsequent state with brakes released without a radio link, were not consistent with any documented fault condition behaviour.
The RCE remained in an unsafe state, which caused the brakes on the train and locomotive to be maintained in a fully-released condition until the train’s brake pipe was ruptured in the derailment.
Appendix B – Safety engineering concepts
Systems engineering
Systems engineering is a way to manage complexity in the development of a product or engineered system. It is a concept that began to form in the 1940s and was later given impetus by the United States National Aeronautics and Space Administration (NASA) as well as the United States military as a way to manage system complexity through control of the development process (Leveson, 2016). The process can be applied to the development of any product.
Leveson (2016) stated:
Combining a systems-theoretic[98] approach to safety with system engineering processes will allow designing safety into the system as it is being developed or reengineered.…A systems engineering approach to safety starts with the basic assumption that some properties of systems, in this case safety, can only be treated adequately in the context of the social and technical system as a whole.
The V-model (or vee model) is commonly used to describe the systems engineering process (for example, see Figure B1). In this model, the product’s user defines a concept of operations or broad set of requirements that describe what the product should do. This description is broken down into a progression of increasingly detailed sets of sub-requirements. The higher-level requirements describe all aspects of what a product should do and how well it should do them, and the lower-level requirements describe the product’s architecture and other detailed elements of design. The requirements drive the design; that is, design elements are chosen based on the best way to meet the requirements. The developers also need to devise ways to prove that the requirements have been met, which forces them to make the requirements verifiable. Safety can be, and often is, among the design objectives.
Figure B1: Systems engineering lifecycle V-model
Source: Systems Safety Assurance Guideline – RISSB, 18 September 2018
Throughout this process, the user needs to be involved to help make the right choices. For example, if a particular requirement cannot be met, or if there are conflicting requirements, the user can decide which path to take based on what is most important.
Designing in this way has many benefits. It helps ensure that the various project goals are well-defined, leaving no important requirement undocumented. This in turn allows important choices to be made about the design early in the process, reducing later rework or undesirable product characteristics.
Once a design is finalised, the design and product passes through increasingly broader sets of verification and validation activities (such as testing) to show that the design does meet the requirements. Verification is the process of showing the final product meets the requirements (that is, the product was built correctly). Validation is the process of showing that the requirements met the user’s needs (that is, the right product was built). If the requirements definition and design process were conducted well, there should be few or no ‘unpleasant surprises’ throughout the verification and validation process.
System integration refers to the progressive assembly of subsystems so that the broader system, as an integrated whole, is able to deliver the overarching functionality. A key component of system integration is defining system interfaces and assessing identified hazards associated with those interfaces. More specifically, as stated by Kusumo (2019):
To ensure safe integration, the SRS [system requirements specification] for a system needs to consider the interface requirements between it and any subsystems and between it and any existing or legacy systems. In addition, these interface requirements for the new system need to include any Safety Related Application Conditions (SRACs) on the existing railway systems that will impact the new system…
A system approach to designing railway systems that will ensure safe system integration involves a systematic analysis of the following:
Interface compatibility between connected railways systems (data, power and signal, etc.);
Risks associated with failures of interface between interconnected systems;
Risks of system failure which may compromise the overall safety of the railway operations;
Compliance with …SRACs from any existing or legacy systems; and
Verification that the identified risk controls have been incorporated in the system design.
Ultimately, the systems engineering process gives the user confidence that, firstly, their needs are well-defined, and secondly, that the product meets them. This is done through, as RISSB states, systematic, methodical, complete, and coherent methods and outcomes. These cannot be achieved without a detailed set of documents that record the process throughout.
Relevant systems engineering standards include:
ISO/IEC/IEEE 15288 Systems and software engineering - System life cycle processes
ISO/IEC/IEEE 24748 Systems and software engineering - Life cycle management.
System safety engineering
System safety engineering can be viewed as the application of systems engineering principles for the purposes of safety assurance (Leveson 2016). It is seen as a more technical process than developing and applying safety management systems but has many principles in common. As with safety management systems or systems engineering, this element can be implemented using a very wide range of different methodologies depending on many factors such as the activity type, organisation size and structure, and external interfaces.
There are numerous system safety methodologies published in Australian and international standards, guidelines, codes of practice, and the like, each with different scope, aims, approaches and/or applications. Further, there is always overlap between them and with other SMS elements such as risk management.
The amount of detail in these standards and guidelines can range from general process-related information (such as how projects can be managed, types of assurance activities, and how these tie into other activities and the system life-cycle) to technical guidance on choosing and performing specific analysis methods (such as failure modes and effects analysis or functional hazard assessment) and numeric methods to estimate the reliability of safety functions.
Safety engineering is often split into 3 parts: system-level, hardware, and software safety engineering. Sometimes, the concept of ‘functional safety’ is applied, whereby ‘safety functions’ that address specific hazards are assigned; it is the reliance on active technical measures to address risk (as opposed to passive methods of addressing risk). ‘Safety integrity’ is a closely related concept, used as a measure of the reliability of safety functions.
Relevant system safety engineering standards include:
IEC/AS 61508 Functional safety of electrical/electronic/programmable electronic safety-related systems
EN 50126 Railway Applications – the Specification and Demonstration of Reliability, Availability, Maintainability and Safety (RAMS)
EN 50128 Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems
MIL-STD-882 Department of Defense Standard Practice: System Safety.
Software safety
Since software fails differently to hardware, many standards (including AS 61508 and EN 50128) provide specialised methods to make software both reliable and safe. The Safety Critical Systems Handbook (Smith and Simpson 2010) recommended that the software safety requirements should include a range of considerations including response times, equipment and operator interfaces, functions which force a safe state, and data corruption, under all modes of operation.
Software errors are not the only reasons software can behave in an unsafe way. Leveson (2011) asserts that:
Nearly all the serious accidents in which software has been involved in the past twenty years can be traced to requirements flaws, not coding errors. The requirements may reflect incomplete or wrong assumptions.
Leveson adds that software may be highly reliable and correct and still be unsafe when viewed from a system perspective, when the system is derived from incomplete requirements, or when it has unintended and unsafe behaviour beyond what is specified in the requirements.
Standards selection and tailoring
Organisations should select standards appropriate to their purpose. For instance, a rail organisation would benefit from using a standard that is developed and endorsed by the rail industry as it is most likely to be a good fit from the outset, and standards sometimes have illustrative examples that are specific to the industry. A small organisation does not have the resources to apply the same level of rigour as a large organisation normally can, and any safety assurance processes would have to be simpler.
An organisation would normally select a methodology that most closely aligns with its functions and structure; this helps maximise the organisational depth of understanding of the methodology, which is critical for its application to be effective.
Having chosen a standard or methodology, it is important for an organisation to understand and define exactly how things will be done in its particular context. The concept of adapting a standard to a particular application is known as ‘tailoring’. The organisation can also adapt its own processes to more effectively align with the chosen methodology.
Some standards provide guidance about how they can be tailored to different applications, often based on enterprise size and complexity.
Moore (2010) advised organisations to:
start small, selecting processes that are coherent, cohesive, widely applicable and capable of adaptation
start simple, selecting a standard that provides varying levels of detail that can be adapted to the situation and with information on where to get more detail
choose standards that provide performance criteria that can be easily understood, and to enable a claim of full or tailored conformance
choose standards that support adding processes, detail, and capability without causing incompatibility.
Relevant standards
Australian Standard 61508
Australian Standard (AS) 61508 is a functional safety standard that provides suppliers of components and subsystems with complete life-cycle processes to optimise system safety through risk- and performance-based measures. It provides a systematic analysis technique to identify hazards and numerically estimate safety integrity, allowing for the capability of a design to detect faults and any limitations in that capability. A system must meet separate requirements for system and hardware integrity to achieve a given safety integrity level.
The base standard (IEC[99] 61508, which is identical) was first published in December 1997 and then adopted as an Australian standard in August 1999.
Under AS 61508, safety requirements are divided into safety functions (functions that defend against hazards) and safety integrity (probabilities of safety functions being successful over time). The safety lifecycles for hardware and software are treated separately but both similarly require:
a safety requirements specification
safety validation planning
design and development processes
integration
operation and maintenance procedures
safety validation
modification, and
verification.
The standard has numerous documentation requirements to support these activities. The standard also lists several dozen ‘recommended’ and ‘highly recommended’ practices to be implemented depending on the product’s intended SIL.
European standard 50126
A European standard issued by CENELEC (the European Committee for Electrotechnical Standardization), EN 50126 Railway Applications – the Specification and Demonstration of Reliability, Availability, Maintainability and Safety (RAMS) provides processes to assure systems-level engineering integrity through 4 overlapping lenses: reliability, availability, maintenance, and safety. It states that it is applicable to all fields within rail (command, control and signalling, rolling stock, and fixed installations) independent of the actual technology of the systems and subsystems. It can apply to new systems, new systems integrated into existing systems, and to some extent, modifications of existing systems.
It comprises 3 volumes:
the generic process,
system approach to safety,
system approach to reliability, availability, and maintainability.
The first volume provides a generic safety management process that is supported by guidance and methods within the other volumes.
The second volume provides guidance and methods for the following areas:
safety process
safety demonstration and acceptance
organisation and independence of roles
risk assessment
specification of safety requirements
apportionment of functional safety requirements (this means finding ways to fulfil the requirements)
design and implementation.
This standard has normative references to related standards such as EN 50128 and EN 50129, discussed below.
EN 50126 has been adopted as a national standard in several countries. An identical standard, IEC 62278, was first published in 2002.
European standard 50128
EN 50128 Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems is a functional safety standard that applies to rail products. It specifies the process and technical requirements for the development of software for use in railway control and protection applications. IEC 62279 is identical.
European standard 50129
EN 50129 Railway applications - Communication, signalling and processing systems - Safety related electronic systems for signalling describes how to show that equipment with the potential to affect safety can be relied on for its functions that relate to safety (that is, how to develop a ‘safety case’). The standard is primarily applicable to railway signalling applications but states that it can also be applied to general-purpose or industrial equipment that is part of a safety-related electronic system. IEC 62425 is identical.
European standard 50657
EN 50657 Railways applications - Rolling stock applications - Software on board rolling stock specifies a process and technical requirements, adapted from EN 50128, for the development of software in rolling stock applications.
Appendix C – Rail Safety National Law National Regulations 2012 (NSW), Regulation 9(1)(a)
Appendix D – Review of relevant Australian Standard 7527 event recording requirements
AS 7527 (Rolling stock event recorders) ‘describes the requirements for event recorders installed in locomotive… rolling stock vehicles’. The standard specified what information was required to be recorded in order to achieve compliance with the standard. The table below provides a review of parameters that were identified as relevant to the 21 September 2018 runaway accident, the requirements detailed in AS 7527, and comment on their application to remote control equipment (RCE) operations.
Event recorder parameter
AS 7527 requirement
ATSB comment regarding application to TasRail cement trains
Vehicle speed, as displayed to the driver
Mandatory
Vehicle speed was recorded at the locomotive. However, this was not always the speed displayed to the driver (when in the driver’s van or operating external to the train).
Position of the throttle or master controller
Mandatory
Throttle response was recorded at the locomotive. The position of the throttle lever was not recorded. The standard contained a further requirement for ‘locomotive distributed power or passenger units operating in multiple’ that the throttle position recorded be the position on the ‘driven locomotive or cab’. While not specifically considering RCE, it did imply that the position recorded was to be that which the driver was directly controlling. When under remote control, this was the position on the RCE transmitter which was not recorded.
Direction of train travel as selected by the driver
Mandatory
Direction of travel was recorded at the locomotive. However, the recorded position was not the position on the RCE transmitter ‘as selected by the driver’.
Brake activation level and duration on the leading driving vehicle
Mandatory
Brake pipe and brake cylinder pressure were recorded on the locomotive. However, the recording could not discriminate whether the RCE receiver and air box or the locomotive airbrake system activating the braking system.
Separately record the operation / application of each braking system fitted to the vehicle
Mandatory
When the locomotive was under conventional (non-remote) control, the position of the (automatic) brake handle was recorded. Correlation between this and the brake pressures (pipe and cylinder) permitted a level of determination regarding which system was responsible for a brake application.
When operated by remote control, the position of the automatic brake lever on the RCE transmitter was not recorded. Therefore, when operated by remote control, there was no indication of which system was applying the brakes, other than through inference from brake pressure response (pipe and cylinder).
Operational status (active / inactive) of fitted ‘deadman’ devices
Mandatory
Operational status of the tilt function fitted to the RCE transmitter was not recorded.
Operational status of the vigilance system (on / off / isolated)
Mandatory
The operational status of the locomotive vigilance system was recorded when the locomotive was under conventional control. When under remote control, a vigilance function on the RCE transmitter was active instead, the operational status of which was not recorded.
Driver acknowledgement of vigilance system alarms
Mandatory
Driver acknowledgement of the locomotive vigilance system was recorded when the locomotive was under conventional control. When under remote control, a vigilance function on the RCE transmitter was active instead, driver acknowledgement of which was not recorded.
All penalty applications of the vigilance system
Mandatory
Penalty applications of the locomotive vigilance function were recorded when the locomotive was under conventional control. When under remote control, a vigilance function on the RCE transmitter was active instead, penalty applications of which were not recorded.
All changes to the operational status of the vehicle (e.g. unmanned automatic / manned automatic / manual)
Mandatory
The operational status was not recorded, other than through inference from other parameters. For instance, if the train was operating while the locomotive was configured in trail cut-out then it was likely under remote control.
All control messages received and transmitted by the vehicle
Mandatory
Control messages received and transmitted between the locomotive, RCE receiver and RCE transmitter were not recorded.
All status messages received and transmitted by the vehicle
Recommended
The status of the RCE receiver and RCE transmitter were not recorded
Main reservoir air pressure, as displayed to the driver
Recommended
Main reservoir air pressure (numbers 1 and 2) were recorded at the locomotive. However, when positioned in the driver’s van, or operating external to the train, the driver did not have these pressures displayed.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
[1] RCE was always used while operating the cement train, including when travelling on the main line and when loading and unloading.
[2] All time references in this report are in local time (Eastern Standard Time).
[3] Each train service on the TasRail network was allocated a unique identifying number. For the cement train, this changed sequentially at Railton and Devonport. Even numbered trains operated from Railton to Devonport, odd numbered trains operated from Devonport to Railton.
[4] That is, 1 m drop in elevation for every 139 m travelled.
[5] For more detail about the transmitter and other components of the RCE, see Remote control equipment.
[6] The stopping mark was a welded indicator on a handrail near the driver’s position in the loading facility, used to visually align the wagons when they were being loaded.
[7] Communication failed mode: a fault mode that occurred when the RCE detected interrupted messages between the transmitter and receiver and was shown to the driver on the transmitter as ‘COMF’. On entry into this mode, the RCE receiver was designed to command the removal of traction power and apply the train’s brakes.
[8] Control fault mode: a fault mode that occurred when a command from the RCE failed to enact correctly and was shown to the driver on the transmitter as ‘CNF’. On entry into this mode, the RCE receiver was designed to command the removal of traction power and apply the train’s brakes.
[9] The action of disconnecting the multiple unit cable resulted in power loss to the RCE receiver, effectively turning it off.
[10] That is, the train was entering a compressive state.
[11] The RCE transmitter (when switched on) was designed to command a brake application if it was tilted at an angle of 50° or more past its normal upright orientation for 10 seconds. This was called the tilt function.
[12] Removing the RCE transmitter battery (when switched on) would normally result in a brake application when the RCE would enter communication failed mode.
[13] Vigilance function: to protect against driver incapacitation; if the driver did not acknowledge a periodic vigilance alarm, the RCE transmitter was designed to command the RCE receiver to apply a full-service brake application.
[14] Three sets of points were facing (onto diverging tracks) and 2 sets of points were trailing (onto a converging track).
[15] Inputs recorded on the driver’s van were limited to the headlight and horn.
[16] Rail transport operator (RTO): describes an organisation that is an accredited rail infrastructure manager or a rolling stock operator or both. TasRail was both the rail infrastructure manager (for example, track, signalling) and rolling stock operator (for example, locomotives, wagons).
[17] Safeworking: an integrated system of operating rules and procedures that define the interaction between workers and engineered systems for the safe operation of a railway.
[18] That is, 1 m drop in elevation for every 139 m travelled.
[19] Selcall: selective calling on a radio system, whereby receiving systems would selectively answer a radio transmission based on the sequential audio tone transmitted.
[20] Driver only operation: a train crewing configuration where a single driver operated the train without the presence or assistance of any other on-board personnel.
[21] For cab handle unit positions and modes of operation, see Train braking systems.
[22] Tests included brake holding, brake pipe leakage, brake inspection, mechanical inspection, loading inspection and brake pipe continuity.
[25] Nadal’s formula is used to calculate the likelihood of derailment. See also RISSB 2014, Derailment investigation and analysis guideline.
[26] Propelling with empty or lightly-loaded wagons between the locomotive and heavily-loaded wagons has a particularly high risk of jack-knife derailment; however, this type of consist was not used in TasRail’s cement train operation.
[27] Parking brakes (or handbrakes) were also fitted to each locomotive and wagon.
[28] This is known as wheel-tread braking: where a brake block applies pressure to the wheel tread, causing friction to dissipate energy, thereby slowing the train.
[29] Train separation: where the rear of the train detached from the front of the train, either through coupler failure or uncommanded uncoupling.
[30] Although different, the driver’s van’s braking components operated in a similar way to that of the cement wagons.
[31] Further braking components were present but have been omitted for relevance and simplicity.
[32] Wheel lock occurs when there is insufficient wheel-to-rail friction to keep the wheel rotating, such as when excessive braking force is applied. In instances of wheel lock the friction of the wheel sliding along the rail head can result in flat spots on the wheel tread. These could result in further wheel damage, rail damage, or derailment where a severe flat spot impacted the rail head once the wheel rotates again.
[33] Draft gear: components that connected the coupling to the rolling stock structure. It also accommodated the dispersal of an amount of buff and draft forces.
[34] Rail burn: damage to the rail head caused by overheating through prolonged wheel slippage. It could result in chips or thermal cracks to the rail head.
[35] This delivered sand to the rail head in front of the leading wheels of each locomotive bogie, increasing friction between the rail and wheels which assisted in a rapid stop.
[36] RISSB 2014, AS7510.1:2014 Braking systems – part 1 – locomotive rolling stock, p. 18.
[37] ATN Tasrail required an RCE unit that allowed flexibility in removal from one locomotive to another, without being semi-permanently integrated into the electronics and pneumatics of the locomotive control systems.
[38] Banking: the use of a bank locomotive provided to assist rail traffic on a steep grade (bank).
[39] For train handling purposes, the brake cylinder pressure from an automatic brake application was usually released on hauling locomotives, without affecting the brake application on the trailing rolling stock. This was known as ‘bailing-off’ the automatic brake application and was achieved by the driver depressing the CHU independent brake handle.
[40] Penalty brake: when induced, reduced the brake pipe to 0 kPa at a service rate, with an emergency alarm sent to the network control centre. The resulting locomotive brake cylinder pressure was about 350 kPa.
[41] Fatal error: a software condition or error that caused the software program to crash (stop running completely).
[42] See Automatic brake for an explanation of the function of these valves.
[43] For a non-remotely controlled train, a driver could detect a brake pipe rupture at the rear of the train using a brake pipe charging flow indicator in the locomotive.
[44] Light engine: one or more locomotives coupled without wagons attached.
[45] The J-multiplier had no effect on a non-remotely-controlled train, because the control pipe would not be pressurised. Therefore, locked wheels would not occur in this situation.
[46] The TR class locomotive would perform an automatic engine shutdown (enter sleep mode) after a 15-minute period of control input inactivity provided certain conditions were met (for example, the train was stationary).
[47] The RCE operation manual provided conflicting advice, stating that the brake pipe pressure must be below 370 kPa during initialisation on page 10, or below 475 kPa on page 27. The ATSB reviewed this section of software code and observed that the actual pressure coded into the software was 470 kPa.
[48] The generation 3 RCE operation manual advised that to reset a control fault mode ‘Reset Transmitter key switch OFF & ON’. There was no mention of required brake pipe and control pipe pressure conditions to be met. See Driver training.
[49] In addition, other problems were identified during operations; see Issues after commissioning and Remote control equipment fault reporting.
[50] The RSNL also stated that the level and nature of responsibility that a party had for rail safety was dependent on the nature of the risk to rail safety that the party created from the carrying out of an activity (or the making of a decision) and the capacity that party had to control, eliminate or mitigate those risks.
[51] EN: European Standard. Further information about EN 50126 Railway Applications – The Specification and Demonstration of Reliability, Availability, Maintainability and Safety (RAMS) is provided in Appendix B.
[52] Under AS 61508, the ‘risk reduction factor’ (allowable probability of dangerous failure per hour) for equipment used continuously was equivalent to an allowable average of 0.1–1 million hours operation without a dangerous failure for SIL 1 and increasing by a factor of 10 for each subsequent SIL, up to 100–1,000 million hours for SIL 4.
[54] This probably means changing a signal path so that it passes through 2 integrated circuits (chips) instead of one.
[55] The GPS speed restriction option consisted of an audible alarm to the driver when the speed limit was exceeded by the train, followed by an emergency brake application and message to the network control centre if speed continued to increase.
[56] This was conversion of the DQ and 2050 class locomotives and generation 2 RCE from a 3-pipe to 4-pipe braking configuration. This added a fourth pneumatic connection between locomotives to allow the release of locomotive brake cylinder pressures induced through a brake pipe pressure reduction.
[57] LED indicators on the RCE transmitter (Figure 13) were automatic brake (TB) and / or independent brake (EB) application in effect, traction power (TH) applied, communication (COM) as a flashing light, low battery (BAT) and dynamic brake (DB) mode active.
[58] Flashover: an electrical arc in an electric motor, sometimes resulting in damage.
[59] TR11 (the accident locomotive) was confirmed to be fitted with the multiple unit pin type which provided the best contact with the receiver’s multiple unit plug.
[60] The ATSB determined this was likely related to the throttle failure incident.
[61] This entailed a change of transmitter, receiver and air box to the spare set, usually in conjunction with a locomotive change.
[62] In its final 3 weeks of operation (up to 9 February 2018), generation 2 RCE had 73 reported events, 63 of which were communication failed mode events.
[63] In March 2018, after ADE advised TasRail the RCE was ‘designed with [several] failsafe features following AS 61508’, TasRail issued a notice of this advice to its drivers, including that the RCE ‘…will only fail safe’.
[64] After the cause of the transmitter screen freezes had been addressed, reported events reduced to an average of about 10 per month, 72% of which were communication failed mode events.
[65] After direction selection failure (21 February 2018), and throttle failure (25 May 2018) incidents respectively.
[66] The network access manager described that a control fault would create an emergency brake application and alarm. While this was the intended response in generation 2 RCE it was not correct for generation 3 RCE as the control fault mode response had significantly changed.
[67] For a description of the designed behaviour refer to the following sections of the report:
the generation 3 RCE control fault mode, see Control fault mode
the generation 3 RCE communication failed mode, see Communication failed mode
conditions which resulted in alarms to the network control centre, see Network control.
[68]Rail Safety National Law National Regulations 2012, Schedule 1 (clause 12 – Management of change).
[69] Routine change was defined in the procedure as those currently ‘…controlled by existing standards, codes, rules, procedures and the SHE management system [SMS]’.
[70] This refers to the independent remote stop mechanism discussed in Independent stop mechanisms.
[71] Fail-to-unsafe: a failure of a system whereby it fails to a dangerous state (for example, the train’s brakes do not apply in the event of a system failure).
[72] FMEA: a systematic approach to document the potential effects of component failures in a system.
[73] ONRSR 2018, The ONRSR Way, pp. 8-9. Version 2 of The ONRSR Way was published in 2020 and the information referenced in this report was effectively the same in both versions.
[74] An accredited RTO had satisfied ONRSR that it had ‘…the competence and capacity to manage the risks, implement the controls and manage changes associated with the railway operations for which it is accredited’. Source: ONRSR 2014, Regulatory response to the management of change policy, p. 9.
[75] ONRSR transitioned to the use of an online portal for some functions in late 2018, after (and unrelated to) the Devonport runaway accident. Among other things, the online portal replaced the previous forms, processes, and guidance related to the notification of change discussed in this report. With respect to the items listed in regulation 9(1)(a), the portal only had guidance for rail infrastructure (item 4).
[77] The term ‘safety critical element’ was not used in any of the other 11 listed items requiring a notification of change. When ONRSR transitioned to the use of an online portal for some functions in late 2018, this definition was no longer included.
[78] AS 4292 Railway Safety Management provided a set of railway safety requirements for incorporation into the management systems of railway organisations. It was published progressively from 1995.
[79] There was no requirement for TasRail to comply with the requirements of AS 7501 or AS 7702.
[80] The requirement for reporting Category A notifications changed after the runaway. From 1 July 2019, significant (Category A) rail safety occurrences were required to be reported to ONRSR.
[81]Rail Safety National Law (South Australia), section 121, Rail Safety National Law National Regulations 2012 (as published on NSW legislation website), regulation 57.
[82] AS 4292.6 was absorbed into AS 4292.1 as part of a revision in 2006. From 2017, Standards Australia no longer maintained the AS 4292 suite of documents.
[83] RISSB n.d., AS 4292 Railway safety management technical note.
[84] FMECA: failure modes, effects and criticality analysis. This is similar to a FMEA but with the addition of quantified probability-severity analysis.
[85]Rail Safety National Law (Tasmania) Act 2012, section 13(2).
[86] Event recorder: a device installed on rolling stock capable of recording multiple input parameters, in digital or analogue format, related to the operation of the rolling stock. Also known as a 'data logger' or 'data recorder'.
[87] A mandatory requirement was defined as a requirement that the standard provided as the only way of treating the hazard. A recommended requirement was defined as one where the standard recognised that there are limitations to the universal application of the requirement, and that there may be circumstances where the control cannot be applied or that other controls may be appropriate or satisfactory.
[88] Often, the term ‘safety case’ is used in a broad sense to encompass all elements of an operation including safety management system(s). In this report, however, the term is used in a more restricted sense, referring to the use of the remotely-controlled cement train application and environment.
[89] Distributed power: the practice of placing locomotives at several locations within a train as distinct from placing all locomotives at the front of the train.
[91] ATSB RO-2018-018 Runaway and derailment of loaded ore train M02712, near the 211 km mark south of Port Hedland, Western Australia, on 5 November 2018.
[92] In Australia, Civil Aviation Safety Regulation 25.001 requires aircraft to meet European or United States regulations, which include the relevant system safety requirements under EASA CS 25.1309 or FAR 25.1309 respectively.
[93] ATSB AO-2008-070 In-flight upset - Airbus A330-303, VH-QPA, 154 km west of Learmonth, WA, 7 October 2008 includes a discussion on the application of safety analysis methodologies within a system safety assurance framework.
[94] See Unintended emergency brake application and release.
[96] The RCE equipment configuration used main reservoir no. 1 air, from the main reservoir equalising pipe (MREQ), to directly supply the brake pipe with pressurised air. The locomotive’s brake system used main reservoir no. 2 air (which was itself fed from MR1 through a one-way valve) to do the same (when in command, see Interface with locomotive). Therefore, the source of a brake pipe pressure recharge (RCE or locomotive) could be determined based on the recorded behaviour of the main reservoir air pressure. If only main reservoir no. 1 pressure decreased, then the recharge was RCE-initiated, as was the case here.
[97] Start position: a configuration in which the RCE transmitter controls were conditioned for a ‘warm reset’ (that is, direction in neutral, throttle idle, independent brake fully applied, automatic brake in at least the ‘initial’ position).
[98] A ‘systems theory’ of safety treats safety as a emergent property of interactions between system components and the environment, allowing it to be controlled by imposing constraints on the interactions (Leveson 2016).
[99] IEC: International Electrotechnical Commission, an organisation for the preparation and publication of international standards for all electrical, electronic and related technologies.
Preliminary report
Report release date: 09/11/2018
This preliminary report details factual information established in the investigation’s early evidence collection phase and has been prepared to provide timely information to the industry and public. Preliminary reports contain no analysis or findings, which will be detailed in the investigation’s final report. The information contained in this preliminary report is released in accordance with section 25 of the Transport Safety Investigation Act 2003.
Overview
At approximately 0846[1] on 21 September 2018, a TasRail freight train rolled away from a loading facility at Railton, Tasmania. There was no train crew on board the freight train at the time. The train travelled for about 21 km before being routed into a dead-end siding in Devonport.
At about 0909, the train collided with the end of a dead-end siding, destroyed a fence line and travelled in a derailed state for about 60 m into a public area (Figure 1). Two pedestrians within the public area received minor injuries from fence debris.
Figure 1: Derailment site in Devonport, Tasmania
Image shows location of injured pedestrians at time of collision and derailment, as well as the configuration of the cement train consist at the derailment site in Devonport on 22 September 2018.
Source: ATSB
Train and track information
The freight train weighed approximately 1132 t and was 220 m long. It consisted of a TR class locomotive (TR11), 16 THFY class bulk cement wagons and a trailing driver’s van (DV1[2]). The freight train was a regular TasRail bulk cement service that operated between the loading facility in Railton and an unloading facility in Devonport (Figure 2).
The track from Railton descends from an elevation of about 58 m (above mean sea level) towards Devonport. Devonport is located on the northern coast of Tasmania and is a shipping port with an elevation of 3 m (Figure 2).
Figure 2: Map of TasRail Network
Image shows location of Railton with respect to Devonport within the TasRail network. Source: TasRail and ARA Railways of Australia 2014, annotated by ATSB.
The TasRail bulk cement service is operated as a driver-alone operation[3] in a push-pull configuration with motive power provided from one locomotive.
To facilitate this push-pull configuration, a portable remote control system is used to control the locomotive. The portable remote control system can be used both from outside of the train driver’s cabin during loading or unloading operations, and from within the leading driver’s cabin when travelling in either direction between Railton and Devonport. The leading driver’s cabin is dependent on direction of travel and can be either from the driver’s cabin in the powered locomotive (TR11), or the driver’s van (DV1) (Figure 1).
Sequence of events
On the morning of 21 September 2018, the train was being loaded at Railton. The driver was situated within the cement loading facility, and was remotely controlling the train to align wagons to the loading chutes. While the last pair of wagons were being aligned, the train came to a stop past the intended stop location.
The driver recalled that, at approximately 0842, he selected reverse to re-align the final two wagons with the loading chutes. However, after selecting reverse, the train became unresponsive to his remote commands.
The driver stated that he attempted multiple times to reset the remote control equipment with the portable remote transmitter. After allowing time for the remote control system to recover, the locomotive continued to be unresponsive to his commands.
The driver decided to walk to the lead locomotive TR11 to undertake a cold restart of the remote control system at the receiver located on the rear of the locomotive. Before he started to walk, at about 0846, the train slowly began rolling away towards Devonport. The driver recalled trying to activate the emergency stop features of the remote system by removing power to the portable remote control system’s transmitter. However, the train did not respond to these commands and gradually gained speed as it rolled away from the loading facility.
At 0848, the driver placed an emergency phone call to TasRail Train Control. He reported the runaway of the cement train to the train controller, who then contacted emergency services.
The Network Access Manager in TasRail Train Control, with the assistance of the TasRail Rolling Stock Assets Manager, were able to monitor the progress of the train using the real time locomotive performance monitoring system. The Network Access Manager communicated this information to the police during the runaway. The police used this information to coordinate their resources to the area ahead of the runaway train. The police concentrated on stopping vehicles and pedestrian traffic at level crossings and about the railway corridor.
During the runaway response, a decision to route the runaway train into the dead-end siding at Devonport was actioned (Figure 3).
Figure 3: Devonport Rail Yard Layout
Image shows route taken by runaway train, and dead-end siding. Source: TasRail, annotated by ATSB.
At about 0909, the runaway train collided with a concrete footing and wooden fence line at the end of the dead-end siding in Devonport. As a result of the derailment and collision, two pedestrians received non-life threatening injuries from fence debris. The locomotive and eight THFY class wagons were significantly damaged.
Post Incident
TasRail advised that its bulk cement service between Railton and Devonport was the only service that used a locomotive remote control system. Following the accident on 21 September 2018, TasRail suspended all use of the remote control system from its operations, pending the conclusion of investigations.
Continuing investigation
The ATSB investigation has conducted interviews with the incident driver, the driver of the train’s previous trip, the rolling stock manager, the injured pedestrians, and the train controller and manager. The ATSB has also gathered documentation about the train, the remote control system and the train network.
Following the accident, a series of tests were organised by TasRail on the locomotive and remote control system. The ATSB were present for a number of these tests.
The ATSB investigation is continuing, and will include consideration of the following:
serviceability of the locomotive, wagons, remote control system, and the interface between them at time of the accident
remote control system and locomotive braking operational inspection and testing arrangements
design and compatibility of the locomotive, remote control system, and the interface between them
implementation management and on-going monitoring of remote control system and locomotive
driver qualifications, experience and medical information
recovery controls:
runaway protection at Railton
detection of unauthorised train access to main line from yard
effectiveness of emergency response systems.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
____________
The information contained in this preliminary report is released in accordance with section 25 of the Transport Safety Investigation Act 2003 and is derived from the initial investigation of the occurrence. Readers are cautioned that new evidence will become available as the investigation progresses that will enhance the ATSB's understanding of the accident as outlined in this update. As such, no analysis or findings are included in this update.
In late 2018 the Australian Helicopter Industry Association (AHIA) requested ATSB assistance to examine cylinders and associated components from the Lycoming engines installed in Robinson 22 and 44 helicopters. This technical assistance was provided to an AHIA sub committee that was examining a reported reduction in engine durability across Northern Australia.
To facilitate this work the ATSB initiated an external investigation under the Transport Safety Investigation Act 2003. The ATSB's assistance included:
provision of a technical advisor to the AHIA
examination of a number of cylinders from affected helicopters in conjunction with the Civil Aviation Safety Authority
review and analysis of engine trend monitoring data from a number of similarly instrumented R22 and R44 helicopters.
This work has been completed and the results provided to the AHIA. Any further inquiries in relation to this issue should be directed to the Australian Helicopter Industry Association.
The information contained in this update is released in accordance with section 25 of the Transport Safety Investigation Act 2003.
On 31 October 2018 a Titanium Explorer Autogyro, registered G-0014, collided with terrain approximately 1 km south‑east of Orange Airport, New South Wales. The pilot and passenger sustained fatal injuries. The rotors and masthead were found some distance from the main wreckage, having separated at the cheek plates (Figure 1). The cheek plates sit on either side of the collapsible mast and are designed to secure the mast in the upright position during operation.
The Australian Sport Rotorcraft Association commenced an investigation into this accident and requested technical assistance from the ATSB to examine the cheek plates and their fracture surfaces. Specifically, the ATSB was requested to determine the direction of fracture progression through the plates and to identify any factors that may have contributed to their failure.
To facilitate this work, the ATSB initiated an external investigation under the Transport Safety Investigation Act 2003.
Results
The ATSB examination undertook physical, microscopic and chemical analysis of the cheek plates. These examinations found that the plates had failed due to ductile overstress, commencing at the leading edge and progressing to the trailing edge (when oriented in the direction of travel). The plate’s dimensions and chemical composition were in accordance with manufacturer’s specifications and there was no evidence of any pre-existing defects.
Figure 1: Port and starboard cheek plates as supplied to the ATSB
Source: ATSB
With the completion of the component examinations, the ATSB has concluded its involvement in the investigation of this accident. Any further enquiries in relation to the investigation should be directed to the Australian Sport Rotorcraft Association.
______________
This report has been released in accordance with section 25 of the Transport Safety Investigation Act 2003.
On 29 October 2018, Lion Air Flight JT610, a scheduled domestic service from Jakarta to Pangkal Pinang, Indonesia, collided with water in the Java Sea, north of Jakarta, about 13 minutes after take-off. The aircraft, a Boeing 737 MAX 8, registered PK-LQP, was destroyed and all 189 passengers and crew on board were fatally injured.
As the accident occurred in Indonesia, the Indonesian National Transportation Safety Committee (NTSC) was responsible for investigating this occurrence. In accordance with Annex 13 to the Convention on International Civil Aviation, the ATSB appointed an accredited representative to the NTSC investigation. The ATSB provided specialist expertise to support the NTSC in downloading and analysing the flight data recorder (FDR) and cockpit voice recorder (CVR) from the aircraft.
The ATSB has concluded its support of this investigation. On 25 October 2019, the NTSC released the final investigation report into this occurrence and it is available at https://knkt.go.id/en
Any enquiries regarding the investigation and report should, in the first instance, be directed to the NTSC.
On 5 November 2018, train M02712, loaded with iron ore, was being operated by BHP on its Newman to Port Hedland railway, Western Australia. The train consisted of 2 locomotives, a rake of 134 wagons, 2 remote locomotives and a second rake of 134 wagons. It was fitted with an electronically controlled pneumatic braking (ECPB) overlay system.
At about 0337, M02712 was travelling at 60 km/h on a downhill grade on the west track, approaching the BHP access road level crossing at the 211.6 km mark. Shortly after, trainline communication between the lead locomotive and the combined end of train monitor was lost, triggering an automated 120% ECPB emergency brake command, stopping the train as it approached Garden South.
Following confirmation with train control of the location of the train and receiving instruction on the number of handbrakes required to secure the loaded train on the falling track grade at Garden, the driver left the locomotive cab to commence applying the brakes from the front of the train. The controller also tasked a support team to attend M02712 and help the driver with applying the handbrakes.
About 60 minutes after the loss of trainline communication, as the driver continued to apply handbrakes to the first rake of ore cars, the train began to move forward. Shortly after, train control received an emergency call from the driver of M02712 alerting that the brakes had ‘bled off’ and the train was now a ‘runaway’.
Train M02712 continued, reaching a speed of 162 km/h before slowing on the rising grades toward Woodstock. After Woodstock, the track grade again began to fall toward Port Hedland and M02712 gained speed to about 130 km/h approaching Abydos.
At about 0520, Hedland control set the crossovers at Turner South and Turner North to switch train M02712 between adjacent tracks to derail the train as it traversed the crossover at speed. About 6 minutes later, the head end locomotives travelling at 144 km/h traversed the crossover at the 119.4 km mark at Turner South.
The locomotives and the first ore car separated from the rest of the train but remained coupled, travelling about 1.6 km further before stopping. The derailment destroyed the 2 remote locomotives, 245 ore cars and 2 km of track infrastructure at Turner South. There was no injury to any person from the runaway or derailment.
What the ATSB found
Between 2011 and 2015, BHP implemented an ECPB system as an overlay to the conventional pneumatic train braking system and undertook an associated modification to the automatic train protection (ATP) system. It predominantly managed the implementation of these changes at an individual system level rather than through the application of a structured engineering approach. BHP did not subsequently identify and manage significant characteristics of how the ECPB, ATP and conventional pneumatic braking systems interacted in response to certain fault conditions. As a result, BHP’s trains configured for ECPB operation were potentially vulnerable to a runaway event should a unique combination of events and conditions occur.
The BHP risk assessment associated with a rail-mounted equipment interaction incident was broad in scope and had limited focus on the causes and critical controls of a train runaway event. In addition, the risk assessment did not include the procedure for responding to brake pipe emergencies and penalties as a critical control. BHP’s material risk control assessments (MRCAs) did not then test the effectiveness of this procedural control for preventing an uncommanded movement of a train during main line operations.
The procedure for responding to brake pipe emergencies and penalties relied extensively on a driver’s memory, with limited processes in place to facilitate or cross-check a driver’s performance to ensure all safety-critical actions were completed. Although the procedure contained a safety-critical action (to apply the automatic brake handle to the pneumatic emergency position), BHP did not clearly communicate the importance and reasons for this action to drivers, reducing the potential for the drivers to correctly recall this action
As M02712 approached Garden on 5 November 2018, one of the 12 trial inter-car connectors disconnected. This caused a loss of ECPB trainline communication and power supply continuity affecting most of the train and triggering an automatic emergency ECP brake application. The driver responded to the ECPB system’s emergency brake application by commencing the brake pipe and penalties and emergencies procedure, but exited the locomotive cab to apply handbrakes to the ore cars without placing the automatic brake handle in the pneumatic emergency position. Without this safety-critical action being done, the brake pipe air pressure was not vented to atmosphere to hold the ore cars brake application via the pneumatic system.
The car control devices (CCDs) on the disconnected ore cars and the end of train monitor continued to run using the internal battery power of each device to hold the brake application. Consistent with how they were designed however, the CCDs released their ECP brake application on shut down after 60 minutes. At this time, while the driver was applying handbrakes, M02712 began to roll away.
The ATP system detected the rollaway and other events, with each generating a penalty brake request to the ECPB system. However, the requests had no effect as the ATP and ECPB systems could not interface to dump brake pipe pressure if an ECPB application became ineffective in arresting an uncommanded train movement.
The ATSB also identified that the response crew tasked to aid the driver did not confirm whether the driver had implemented the BHP three-step protection process prior to approaching a train to begin the application of handbrakes. This increased the risk of injury to personnel working on the rolling stock. Additionally, following becoming aware of the runaway, the BHP emergency response procedures did not ensure rail infrastructure managers that interfaced with the BHP rail network were alerted to an emergency event that could affect safety at the interface.
Given that the train stopped at 0340 and the driver was conducting a series of 7 night shifts, the ATSB examined BHP’s processes for managing train driver fatigue. The ATSB found that the BHP roster patterns for fly-in fly-out train drivers were conducive to result in cumulative sleep restriction and levels of fatigue likely to adversely influence performance on a significant proportion of occasions, and BHP had limited processes in place to ensure that drivers actually obtained sufficient sleep when working these roster patterns. Due to cumulative sleep restriction over several days of night shifts, the time of day (0340) and other factors, the driver of M02712 was probably experiencing a level of fatigue known to adversely influence performance. However, based on the available evidence, the ATSB did not conclude that fatigue contributed to the runaway of M02712.
What has been done as a result
Following the runaway and derailment accident involving M02712, BHP reviewed the risk management framework associated with rail-mounted equipment interaction, updated the risk assessment, and added additional controls related to potential train runaway events. Additionally, BHP implemented a systems engineering and assurance framework to manage the future integration of systems utilised within the BHP rail system.
With regard to procedural controls, BHP revised its operating instruction for responding to brake pipe emergencies and procedures by requiring the driver to complete a form confirming the actions undertaken in response to an emergency ECPB application and confirming these actions with train control prior to leaving the locomotive cab. In addition, the operating instruction was amended to clearly advise the importance and rationale for drivers to place the automatic brake handle in the pneumatic emergency position in response to an emergency ECP brake application with the end of train monitor displaying ‘off’ or ‘?’.
BHP also revised the work instruction associated with handbrake application and release during main line recovery to require that a work group supervisor be appointed to communicate directly between the driver and the work group tasked to render assistance.
BHP has commissioned external fatigue subject matter experts to undertake a range of evaluation and development activities. BHP has recognised that its roster design was not conducive to minimising fatigue and has formed a working group to optimise rosters. It has also undertaken additional work to improve fatigue training and fatigue monitoring of drivers.
Safety message
A train runaway can cause injury or loss of life, substantial damage to rolling stock and infrastructure, and disrupt rail operations for an extended period. Rail transport operators should therefore ensure that they conduct thorough risk assessments to ensure that relevant causes and hazards associated with runaway events are identified and managed.
In addition, rail transport operators considering changes involving the integration of complex systems should utilise a systems engineering approach to identify hazards and then manage risk to ensure that the railway’s operations remain safe, so far as is reasonably practicable. Rail transport operators must then ensure the preventative controls mitigating the hazards will be effective in managing the risk. They also need to place adequate emphasis on critical controls to signify their importance and ensure that the rail safety workers who are required to implement procedural controls clearly understand why the specified actions are required.
Emergency procedures communicate critical tasks that must be fully actioned by rail safety workers responding to atypical or unexpected situations. Rail safety workers must therefore ensure they take sufficient time to methodically perform and verify the effectiveness of each required action.
The occurrence
Overview
On 5 November 2018, train M02712, loaded with iron ore, was being operated by BHP on its Newman to Port Hedland railway, Western Australia, from Mining Area C to Nelson Point (Figure 1).
At about 0340,[1] the train stopped at the 210.7 km mark near Garden South due to a loss of trainline communications. As part of the response to this fault, and in discussion with network control, the driver exited the locomotive and commenced applying the handbrake on each ore car. At approximately 0440, with the driver still applying handbrakes, the train rolled away. There was no driver on board the train at the time.
The train travelled uncontrolled on the west track for about 91 km before Hedland train control derailed the train by routing it from the west track to the east track at a crossover located at Turner South. At about 0526, the head end locomotives traversed the crossover. Shortly after, 245 ore cars and the 2 remote locomotives, positioned mid train, derailed while travelling at 144 km/h.
Figure 1: Map of BHP’s Newman to Port Hedland railway
Source: BHP, annotated by the ATSB
Events prior to loss of trainline communications
At about 2300 on 4 November 2018, train M02712 departed Mining Area C (Figure 1) for the driver exchange point, M308 (308 km mark). It consisted of 2 locomotives leading, a unit rake of 134 ore cars, 2 remotely-operated locomotives located mid train, and a second unit rake of 134 ore cars. The trains brake control system was set to enable electrically controlled pneumatic brake (ECPB) operation.
Following arrival at M308, the next rostered driver for the train boarded and took over control of M02712. The train departed the driver exchange point at about 0115, crewed in a driver-only configuration.
At about 0337 on 5 November 2018, M02712 was travelling at 60 km/h on a downhill grade on the west track, approaching the BHP access road level crossing at the 211.6 km mark. The driver had set the throttle control for maximum dynamic braking to control train speed for descending the grade and began moving the automatic brake handle toward a 39% train brake command (TBC).
At about 0338, electrical communication via the trainline between the lead locomotive (4420) and the end of train monitor (EOTM) was lost, triggering an automated emergency brake application (120% TBC), stopping the train at about 0340 as it approached Garden South.
Response to loss of trainline communications
In response to the loss of trainline communications, the driver fully applied the locomotive independent brake. The driver made an emergency radio call to Hedland train control to report the occurrence, the location (at the 210.737 km mark between Shaw and Garden), and the detail of the alert messages displayed on the locomotive onboard systems.
The train controller placed blocks to the trackside signals on the adjacent east track between Garden South and Shaw North[2] to protect the train from other rail movements, and contacted personnel from the Redmont[3] maintenance gang to assist the driver. The controller told the driver that help was on the way and requested the driver to confirm the train’s location from a kilometre mark on the ground closest to the lead locomotive. The driver advised the controller that the FIRE[4] system displayed 210 km but they would detrain and check the kilometre mark to confirm.
At about 0350, the driver confirmed to the controller that the train was stopped at the 210.7 km mark. The controller then instructed the driver to apply 101% handbrakes[5] to secure the loaded train on the falling track grade at Garden. The controller asked if the driver wanted to start applying them now or wait in the locomotive for the arrival of personnel from the Redmont gang (who had been tasked by the controller to check the rear of the train before starting to apply the handbrakes from that end). The driver advised that they would start applying the handbrakes to the first rake of 134 ore cars rather than wait for the Redmont gang to arrive.
At about 0351, the driver placed the reverser[6] control to the centre (neutral) position and turned the generator field[7] off before preparing to exit the cab of locomotive 4420. The 120% emergency TBC was still active and the automatic brake handle remained set at the position equating to a 39% ECPB TBC.[8] The driver was aware the emergency ECPB interlock[9] would maintain the ECPB application and had applied the independent brake to secure the train. Additionally, by placing the reverser to the centre position and turning the generator field off, the driver had set up the locomotive rollaway protection provided by the on-board automatic train protection (ATP) system.
However, the driver did not place the automatic brake handle in the pneumatic emergency position to vent the train brake pipe pressure to atmosphere. This meant an additional braking control via the conventional pneumatic train brake system was not activated, and the emergency brake application was maintained by the ECPB system only.
Application of handbrakes
At about 0353, the driver exited the locomotive and commenced applying handbrakes to the ore cars, starting from the front of the train.
Soon after starting to apply the handbrakes, the driver identified that one of the inter-car trainline cable connectors near the front of the train had detached. The trainline was located on the opposite side of the train to the handbrake controls. As required by the operator’s procedures, the driver continued applying handbrakes to the remaining ore cars. The driver recalled that the process of applying handbrakes was relatively slow due to the difficulty climbing up and down the steep-sided ballast formation next to each ore car in the dark.
Train control continued to keep in contact with the driver of M02712 via the driver’s handheld radio at 10-minute intervals. During the first of these scheduled calls, the driver advised of finding the disconnection in the trainline cable.
At about 0422, personnel from the Redmont gang informed Hedland train control of their arrival at the 210 km mark to aid the driver of M02712 with applying handbrakes. The controller tasked the gang to start applying handbrakes from the rear of the train and continue toward the driver, who was working from the front. The controller also requested that, when the Redmont gang reached the rear of the train, they provide a report on the integrity of the rear of the train.
During a subsequent scheduled 10-minute call with train control, the driver reported to the controller that the application of handbrakes was progressing well, despite having trouble walking along the elevated ballast shoulder next to the stationary train. The driver stated they were about ‘three quarters’ of the way toward the remote locomotives in the middle of the train. They also reported being aware that the Redmont gang would check the integrity of the rear of the train and start applying handbrakes from there. The driver advised the controller that they intended to continue working toward the remote locomotives in the middle of the train, report to train control, and then return to reinstate the break in the trainline cable. The controller stated that, as the driver and maintenance gang were now ‘both on the ground’, they could communicate with each other.
The driver later advised they had formulated their plan to apply handbrakes on the first rake and then go fix the connector on the assumption that the personnel in Redmont gang, sharing the task of applying handbrakes, would be able to move at a faster rate; meaning that they would arrive at the remote locomotives at about the same time as the driver.
The runaway
At 0438, 60 minutes after the loss of trainline communications, the brakes released on most of the ore cars in the train. At this time, the driver was still applying handbrakes to the first rake of ore cars, and they recalled that they were about 20–30 ore cars from the rear of the first rake. The driver initially heard air venting from the ore car brakes and shortly after noticed the train lurch forward and start to roll away. The driver recalled that they tried to radio the Redmond gang and alert them that the brakes had ‘bled off’ but there was no response.
Shortly after train M02712 began to roll away, the ATP system detected the movement and requested a penalty brake application, but it was ineffective in stopping the train.
Previously, at about 0355, an empty ore train (M02727), travelling on the adjacent east track toward Yandi Junction, stopped at Garden South due to the blocking protections set up previously. At about 0444, the driver of this empty ore train contacted Hedland train control to advise that M02712 was moving and had passed Garden South at an estimated speed of about 50 km/h with brakes dragging.[10]
At 0446, train control received an emergency call from the driver of M02712, stating that the brakes had bled off and the train was now a ‘runaway’. The driver of M02712 had lost their footing when the train began to move, slipping on the ballast formation and knocking their radio off channel. After resetting the radio, the driver contacted the train controller, declaring an emergency and notifying of the runaway. Train control acknowledged the emergency call and advised that signal GNN4 at Garden North was set to red, in order to stop the train by triggering the locomotive onboard ATP system.
Train M02712 passed signal GNN4 at about 80 km/h and continued to increase speed. Although the ATP system requested a penalty brake application in response to signal GNN4 at red and to an overspeed condition, these penalty requests were also ineffective in stopping the train.
About 80 km ahead, another train (M02728) travelling on the eastern track was approaching Abydos North. Hedland train control contacted its driver, instructing the driver to stop, detrain and move to a safe place. Train control also contacted the drivers of the 2 other trains (M02729 and M02710) working between Garden North and Port Hedland, instructing them to also stop, detrain and move to a safe place. Trains M02729 and M02710 stopped at locations north of Turner (Figure 1).
At about 0502, the driver of the empty ore train stopped at Garden South (M02727) contacted train control to advise that the Redmont gang had mistakenly started applying handbrakes to their train rather than to M02712.
Train M02712 continued through Spring and Coonarie. It reached a speed of 162 km/h before slowing on the rising grades toward Woodstock (Figure 1).
At about 0509, M02712, travelling at about 128 km/h, passed over the active level crossing at the 154.3 km mark before Woodstock South. After Woodstock, the track grade again began to fall toward Port Hedland and M02712 gained speed to about 130 km/h as it passed M02728 stopped at the 130.5 km mark on the eastern track north of Abydos.
The derailment
At about 0520, Hedland train control set the crossovers at Turner South and Turner North to switch the runaway train M02712 between adjacent tracks to derail it as the train traversed the crossovers at speed.
About 6 minutes later, the head end locomotives, travelling at 144 km/h, traversed the crossover at the 119.4 km mark at Turner South. Locomotives 4420, 4434 and the first ore car separated from the rest of the train but remained coupled, travelling about 1.6 km further before stopping (Figure 3). The first ore car had derailed.
Ore cars in position 2 to 134 of the first unit rake, the remote locomotives 4472 and 4440 and ore cars one to 112 from the second unit rake derailed near the crossover (Figure 3). The last 22 ore cars of the second unit rake remained coupled and on track.
The derailment destroyed the 2 remote locomotives, 245 ore cars and about 2 km of track infrastructure at Turner South (Figure 4). There was no injury to any person from the runaway or derailment.
Figure 2: Locomotives 4420, 4434 and first ore car at Turner South
Image viewed in a southerly direction of locomotives and ore car at Turner South. The lead locomotives 4420 and 4434 and one ore car remained upright, however the ore-car had derailed.
Source: BHP, annotated by the ATSB
Figure 3: Train M02712 wreckage near the crossover at Turner South
Aerial image viewed in a southerly direction of train wreckage and track damage at Turner South. The lead locomotives 4420 and 4434 remained on track and were coupled with one ore car that had derailed (out of frame in the foreground).
Source: BHP, annotated by the ATSB
Figure 4: Ore cars and remote locomotive 4427 wreckage at Turner South
Wreckage of remote locomotive 4472 and ore cars from rakes A and B viewed in a south-westerly direction at Turner South.
BHP was the rail infrastructure manager for the Newman to Port Hedland railway, which it used to transport iron ore. The railway was a standard gauge track structure constructed with continuously welded 68 kg/m rail, fastened with resilient clips to concrete sleepers bedded in crushed rock ballast. The track structure configuration enabled the operation of rolling stock with a 40-t axle load.
In the direction of travel, the track gradient from Mining Area C was primarily a rising grade approaching Shaw in the Chichester Range, before transitioning to a mainly falling grade toward Nelson Point (Figure 1). The M02712 runaway started between Shaw North and Garden South where the track gradient was -1.5%, the steepest track gradient of the track section between Yandi Junction and Nelson Point.
An automatic train protection (ATP) system governed the maximum permissible track speed for the various sections dependent on the mode of operation (loaded or unloaded), with the target speed displayed to a driver via the FIRE system. The maximum track speeds for the Newman to Port Hedland railway were 60 km/h for a loaded and 75 km/h for an unloaded ore train.
Train control information
BHP managed train movements remotely from a train control centre located in its integrated remote operations centre in Perth. The train control centre had 5 operational control areas (desks): Hedland, Newman, 6PG, Hub control and Yard control. All communications between the Perth control centre, train movements, control systems and wayside equipment was via a dedicated VHF radio system.
The runaway occurred within the operational area managed by Hedland train control, which extended from the 67 km mark south of Walla to the 260 km mark south of Cowra (Figure 1).
Train crew information
Qualifications and experience
The driver of train M02712 began employment with BHP in 2008, operating ore trains from Port Hedland and later the Yandi depot. The driver had recently completed the BHP driver reaccreditation in driver safeworking, locomotive system theory and in-field training courses, and held the required competencies for the tasks performed (see also Driver competency assessment related to rules and procedures).
Medical information
The driver underwent their last medical assessment (category 1) on 9 May 2018 and was assessed as fit for duty as per the requirements of the National Standard for the Health Assessment of Rail Safety Workers.
Following the occurrence, BHP initiated screening tests on the driver for the presence of an illicit drug or alcohol, which provided a negative result (that is, no alcohol or drugs were detected).
Recent history
The driver was employed under a fly-in fly-out (FIFO) arrangement. BHP’s train drivers working on a FIFO arrangement typically worked a roll-over roster pattern or ‘swing’ that included 7 12-hour shifts (each starting at the same time of day), a 24-hour recovery break, 7 12-hour shifts (each starting at the same time of day), and 12 days off duty.
The driver commenced a roll-over swing on 31 October 2018, as outlined in Table 1. This involved commuting to Adelaide on 29 October, staying overnight in Adelaide, then commuting during the day from Adelaide to Yandi on 30 October. The driver recalled waking at about 0230 Western Standard Time[11]to make an early flight to Perth, then catching an afternoon flight to Newman (during which they had a short nap) and arriving at the Yandi depot at about 1700. They had 4–5 hours after arriving at Yandi to allow for unpacking, getting a meal, going back to their accommodation, checking in with family and obtaining some rest prior to preparing for work.
The driver then commenced the series of 12-hour night shifts, each starting at 2200. Each shift generally involved taking a loaded train from Yandi to Port Hedland or taking an empty train from Port Hedland to Yandi.
Table 1: Scheduled and actual duty times for the driver of M02712
Date
Work activity
Duty start
Duty end
Duty time
Time free (of duty)
28 Oct 2018
Day off (11th day free of duty)
29 Oct 2018
Day off
Commute to Adelaide (1600–2000)
30 Oct 2018
Commute Adelaide – Yandi depot (0230–1700)
Yandi–Port Hedland
2200
1000
12 hours
12 hours
31 Oct 2018
Port Hedland–Yandi
2200
1000
12 hours
12 hours
1 Nov 2018
Yandi–Port Hedland
2200
1000
12 hours
12 hours
2 Nov 2018
Port Hedland–Yandi
2200
1000
12 hours
12 hours
3 Nov 2018
Yandi train load out
2200
1000
12 hours
12 hours
4 Nov 2018
Yandi–Port Hedland (train stopped at 0340)
2200
1000
12 hours
12 hours
5 Nov 2018
Night shift (planned, not worked)
2200
1000
12 hours
24 hours
6 Nov 2018
(Finish duty at 1000)
7 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
8 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
9 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
10 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
11 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
12 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
13 Nov 2018
Day shift (planned, not worked)
1000
2200
12 hours
12 hours
All times in the table are in Western Standard Time (UTC + 8 hours). The driver commenced commuting on 30 October from Adelaide, which was 2.5 hours ahead of WST.
The driver signed on for their sixth night shift on 4 November at about 2200 after a short (10–15 minutes) commute. The driver recalled that there was a delay in their loaded train being ready for departure. They eventually departed the driver exchange point, M308, at about 0115. They were running behind another train and experienced some yellow signals and reduced speeds, but had no stoppages until the train stopped near Garden at 0340.
Drivers were entitled to a 30-minute ‘crib’ break (or meal break) each shift, and the driver expected to get their break at about 0430–0500 due to the late departure (and breaks normally being taken with a train on a flat gradient). The driver reported that, in the period leading up to the train stopping, the workload and complexity were both moderate (which was normal for that location).
The driver stated that, after stopping work at 1000 following a night shift, they would go back to their accommodation and normally slept for about 4–5 hours. They then had a meal when the kitchen opened at about 1700, watched television, made a call home and then dozed or napped (if sleep occurred). Overall, they would normally get about 5–6 hours sleep each break following a 2200–1000 shift, with some of this being broken sleep and sometimes sleep being difficult to obtain. This contrasted with 7 plus hours of good quality sleep from about 2200 to 0630–0700 when they were at home.[12]
The driver reported that they found swings commencing at 2200 the most difficult for obtaining sleep. The first few days were particularly difficult, and they would get more sleep later in the week. However, they would continue to get less sleep each rest period than they would at home.
The driver self-rated their fatigue level at the time of the occurrence as 4 out of 7 (‘a little tired’) [13]but also noted that a person usually feels better than they actually are. The driver also commented that, after the train stopped, they felt ‘deflated’ when they realised they had to walk alongside the train to secure the handbrakes.
In terms of strategies to maintain alertness, the driver stated they drank some coffee each day during a swing, mainly while waiting for their train to be ready rather than on the train. They would also listen to music when in the train. They did not take any medications to maintain alertness or drink alcohol during a swing.
Further information regarding BHP’s fatigue management procedures are provided in Fatigue management.
Train information
General information
BHP’s ore trains ran as unit trains.[14] Train M02712 consisted of 2 SD70ACe type locomotives (4420, 4434) leading, a unit rake of 134 ore cars, 2 remotely-operated SD70ACe type locomotives (4472, 4440) located mid train, and a second unit rake of 134 ore cars. It weighed approximately 42,500 t and was 2,860 m long.
The ore train was working between the loading facility at Mining Area C, situated on the spur line extension from Yandi, and the unloading facility at Nelson Point, Port Hedland (Figure 1).
SD70ACe type locomotive
BHP’s SD70ACe diesel electric locomotives were equipped with a microprocessor-based computer control system (EM2000). This control system monitored and controlled locomotive traction power, braking and other interfacing systems. The control system detected fault conditions and allowed diagnostic testing of associated systems. The interface between the locomotive control system and driver was through the functionally integrated railroad electronics system (FIRE).
The FIRE display panel (or integrated functional display) was located in the driver console. The FIRE system replaced most of the driver control switches, gauges and indicators with a display panel graphic user interface. The display screens provided an interactive system that allowed viewing of pertinent data and provided input signals to the locomotive control and air brake systems for set-up and diagnostic tasks. The system also displayed information on an event basis, such as alarms and operator crew messages. ECPB set-up and other functions were performed using various ECPB menus on the display panel.
Some locomotive control functions that previously operated independently through their own display screens were integrated into the FIRE display console. Such functions included those associated with the:
EP-60 brake controller
ATP system
locomotive microprocessor control system
ECPB system.
Overview of BHP ore car braking system
The braking application on a conventional pneumatically-braked ore car relied on the driver operating the automatic brake handle in the locomotive to generate a reduction in brake pipe pressure (pressure wave) within the brake pipe. The pressure wave propagated along the brake pipe (through each ore car) for the length of the train. This pressure wave actuated a pneumatic brake control valve in each ore car, applying the ore car’s brakes. The ore car brake applications occurred sequentially along the train, with the magnitude of the brake effort determined by the reduction in brake pipe pressure made by the driver.
Conventional pneumatic braking systems therefore allowed the driver to graduate the application of braking effort on the train. These systems required the brake pipe to be fully charged before the pneumatic valves in each ore car would release the brake application. This meant that the driver could not graduate the release of a brake application.
In contrast to conventional pneumatic-braking systems, an electronically controlled pneumatic braking (ECPB) system used electronic signals that made it possible to activate the air-powered brakes on each ore car. To facilitate this, the ore cars were equipped with a trainline cable that ran parallel to the brake pipe along the length of the train. The cable supplied power to the electronic components installed on each ore car. The cable also served as a communication medium that allowed the locomotive control system to send commands and receive feedback from the ore cars and the end of train monitor (EOTM).
ECPB provided benefits over the conventional pneumatic braking system. For example:
Since all the ore cars received the brake command at the same time, the brakes were applied uniformly and instantaneously. This minimised in-train forces and provided better train control, shortened the stopping distance, and reduced risk of derailment or of coupling breakage.
The brake pipe remained charged during a brake application. This allowed the reservoirs on the ore cars to continuously charge with air.
As the ore cars could send their status to the locomotive at the front, ECPB provided better diagnostic capabilities and enabled the train crew to better monitor the state of the train and its braking capabilities.
ECPB trains could take 2 forms: stand-alone or overlay. With a stand-alone system, braking controls on the ore cars would only operate electrically and there was no pneumatically-operated valve installed on the ore car. An overlay system essentially retained the pneumatic valve and added the electronic car control device (CCD), electronically-controlled valve and adapter air manifold (Figure 5).
Figure 5: Typical ore car air brake system with ECP brake overlay
ECPB system highlighted in green.
Source: BHP, annotated by the ATSB
BHP selected an overlay option to provide operational flexibility in mitigating delays to production should an ECPB train experience a fault that could not be recovered in a timely manner. The overlay system allowed BHP to operate trains as either a conventional pneumatically-braked train or as an ECPB train.
In 2011, BHP commenced a management of change process for the introduction of the ECPB system to its mainline operations (Management of change). It then progressively implemented the ECPB system throughout its fleet.
Braking and distributed power systems on train M02712
Train M02712 was equipped with an EP-60 New York Air Brake (NYAB) ECPB system. The system consisted of locomotive equipment, ore car braking control equipment, an EOTM, and a power and communications distribution system.
Locomotive equipment included a trainline communications controller, power supply and identification module. The head end unit (HEU) locomotive communicated with each of the 268 CCDs and remote locomotives via embedded transmissions in the trainline cable, comprised of a single pair of wires forming the intra-train power and communications network. The trainline cable between each rail vehicle (locomotive or ore car) was joined using a connector (see also Trial trainline inter-car connectors). Each CCD unit used 230 V direct current power from the trainline cable to charge its batteries and supply power to its electronics.
The EOTM installed on the last ore car coupler marked the end of the train. It also provided a termination point for the trainline cable and a transducer for end of train information, such as brake pipe pressure, back to the HEU to establish the integrity of the trainline and train consist. The EOTM used 230 V direct current power from the trainline cable to charge its batteries and supply power to its electronics.
If the power from the trainline cable was lost, the CCDs and EOTM each continued to operate on battery power until a 60-minute time period elapsed (shut-down mode or battery conservation mode – refer to ECPB codes of practice and standards) or the battery charge ran low and the CCD cut out. The CCDs and EOTM then respectively entered the shut-down mode or cut out. When a CCD shut down or cut out, it released its ECPB application and relinquished control of brake cylinder pressure to the conventional pneumatic braking system of the ore cars. If the brake pipe was charged and a pneumatic application was not in effect, the brake cylinder pressure released.
BHP was not able to advise on the average battery life of a CCD or EOTM battery, but it was understood to be substantially longer than 60 minutes.
The BHP locomotive fleet was equipped to enable control of multiple distributed power units within the train. Communication of synchronous control and indication signals between the HEU, trailing and remote locomotives also occurred via the trainline system.
As a contingency, the ECPB overlay system and trainline could be shut down and the HEU configured to communicate power and brake commands via UHF radio communications to the remote locomotives. This configuration disabled ECPB, and train braking reverted to conventional pneumatic operation via the train brake pipe. The HEU configuration also set up communication with the EOTM by radio.
The FIRE system displayed braking parameters related to the ECPB system mode, alarms, diagnostic messages, and brake command input (Figure 6). The system displayed the level of train brake command (TBC) input as a percentage, typically between 0% and 100% or as 120%. Various values meant:
0% = release
10% = minimum service
100% = full service / penalty application
120% = emergency.
Figure 6: Typical FIRE system display
Typical parameters displayed on FIRE system display monitor. Details shown were not those present on locomotive 4420 at the time of the M02712 runaway.
Source: BHP
Driver electronic brake control unit
The driver could manually control braking applications through the electronic brake control unit located on the driver’s console. The control unit included the automatic and independent brake handles. Each of these handles provided independent electrical signals to the EP-60 brake controller (Figure 7).
Figure 7: Drivers electronic brake control unit
Position of the automatic brake handle equates to a 39% TBC brake application.
Source: BHP, annotated by the ATSB
The automatic brake handle had the following detent positions for driver control:
REL (release) – charged air brake and releases locomotive and train brakes
MS (minimum [service] reduction) – first detent in service zone to apply minimum braking
Service zone – between MS and FS applying graduated service braking effort
FS (full service) – position in service zone to apply full-service braking effort
SUP (suppression) – second detent position applying full-service braking effort and suppression to safety control applications
HO (handle off) – third detent position used when driving station is not active (handle is not removable) or 120% TBC when driving station is occupied and operating in ECPB mode
EMER (emergency) – fourth detent position, brake pipe pressure reduced to 0 kPa at a rapid rate and when configured to ECPB mode applied 120% TBC.
In addition to the driver controlling braking to the train (locomotives and ore cars) via operating the automatic train brake handle, the driver could control braking to the locomotives via operating the independent brake handle. The independent brake handle was directly below the automatic brake handle and controlled the HEU locomotive’s braking independently of the automatic train brake (Figure 7). It also applied the brakes on other locomotives in the train (lead and trail) but it did not apply brakes on the ore cars or remote locomotives. The independent brake control applied brakes pneumatically, irrespective of the HEU configuration.
The independent brake control handle could be positioned to:
REL (release) – released the locomotive brakes, if the automatic brake handle was also in the REL position
SERVICE – moving the handle through the service zone increased locomotive braking effort
FULL – applied full braking effort on the locomotive(s)
bail off function – depressing the handle in either the REL position or SERVICE zone suppressed any automatic train brake application in progress on the locomotive(s).
Figure 7 shows the position of the brake handles at the time of the runaway and derailment. The driver positioned the automatic brake handle in the service zone to the position equating to a 39% ECPB TBC at 0337, prior to the loss of trainline communications at 0338. The independent brake handle was moved to the FULL position at 0353.
ECPB codes of practice and standards
The Rail Industry Safety and Standards Board (RISSB) Code of Practice for ECP braking (released in 2017) described the configuration and operation of trains fitted with ECPB for use in the Australian rail industry. The practices described in the code were recommendations to the rail industry but excluded captive unit train operations[15]or situations where rolling stock operators, vehicles or locomotives did not interchange across functional boundaries.
The code referred to content from the suite of documents published by the Association of American Railroads (AAR) under Section E-II - Manual of Standards and Recommended Practices - Electronically Controlled Brake Systems. The AAR adopted the S-4200 standard in 1999 with later revisions in 2002, 2004, 2008 and 2014. The standard defined the requirements to ensure the functionality, performance and interoperability for an approved freight train power brake using ECPB systems.
The standard specified the normal operation functions of an ECPB system (including an overlay system) and addressed the effect of the CCDs or EOTM entering the shut-down mode following loss of trainline power. Section 4.3.17 stated:
Shutdown mode (or “battery conservation” mode) shuts off the CCD or EOT to minimize battery drain. When shut down, the CCD or EOT is turned off. When a CCD shuts down, it releases its ECP brake application and relinquishes control of brake cylinder pressure to the pneumatic backup. If the brake pipe is charged and a pneumatic application is not in effect, brake cylinder pressure will release; otherwise it will remain at the level commanded by the pneumatic backup. When an EOT shuts down, it stops transmitting EOT beacons.
Once train line power is lost, the CCD or EOT either continues to operate off of battery power until its battery runs low or enters into a timed shutdown mode. The intent of this logic is to allow the train to operate as long as possible after a loss of train line power and to conserve batteries if the device is disconnected from the train line, the train is parked, or the ECP brake system is CUTOUT.
The CCD or EOT shall shutdown 1 hour after both train line power is lost and no HEU beacon has been received. The CCD or EOT shall shut down after the train line power is lost and the train operating mode is set to CUTOUT.
The standard detailed conditions that triggered the designed shut-down mode of operation. Although the functionality supplied flexibility in operation and protected battery condition under certain circumstances, such as shunting operations utilising switch mode,[16] the conditions also existed following an interruption to the trainline during main line operations.
The standard stated that the CCD on each ore car downstream from a trainline interruption would release the ECP brake application and relinquish control to the pneumatic backup system 60 minutes after a loss of trainline continuity. If the CCDs were shut down with the brake pipe air pressure charged, the brake cylinder pressure on the affected ore cars would release and the braking effort would be lost.
In summary, the operation of the ECPB system outlined in the S–4200 standard included a 60-minute shut-down feature of the CCDs in certain conditions, which introduced a potential risk exposure for a runaway event that needed to be managed. The ECPB system on BHP’s trains was designed consistent with the standard.
ECPB emergency brake conditions
In the ECPB mode of operation, the initiation of an emergency braking application could occur:
automatically in response to various system-detected conditions
manually by the driver moving the automatic brake handle to the ‘handle-out’ or pneumatic ‘emergency’ positions.
If the system detected an emergency brake condition, such as a critical loss of trainline communications, the EP-60 manual described the system response as:
Emergency (120%) brake command, an emergency brake (120%) brake interlock, locomotive power knock-down (PCS) and corresponding crew message(s). The 120% emergency brake interlock will remain in effect for a minimum of 2 minutes since the emergency condition occurred. The interlock can then be reset once the condition that caused the emergency has been corrected.
If a driver triggered the emergency application by moving the automatic brake handle to the pneumatic ‘emergency’ position, the EP-60 manual described the functionality:
In this position brake pipe is vented to zero and a 120% TBC train brake command is provided.
If the driver moved the automatic brake handle to the ‘handle-out’ position, the braking response would be the same as for the system-initiated emergency (120%) brake command. As stated in the EP-60 manual:
If the automatic handle is moved to the “handle-out/continuous service” position, the brake pipe will continue to charge but a 120% TBC train brake command is provided.
In ECPB mode, for a system-initiated emergency brake application or when a driver moved the automatic brake handle to the ‘handle-out’ position, the 120% TBC brake interlock feature maintained a full air brake application to the train, but the brake pipe air pressure remained fully charged at 600 kPa. For a driver-initiated emergency application where the driver moved the automatic brake handle to the pneumatic emergency position, both the 120% TBC brake interlock and the discharged brake pipe maintained a full air brake application on the train.
Emergency brake application on train M02712
In the case of M02712 approaching Garden on 5 November 2018, the driver had positioned the automatic brake handle for a 39% train brake call to the EP-60 brake controller, prior to the emergency (Figure 7).
About 30 seconds later, at 0338, the interruption in trainline continuity caused several critical alarm conditions in the HEU locomotive’s onboard systems. The loss of trainline communications between the EOTM and HEU beacon triggered a system-initiated emergency application of 120% TBC to lead locomotives and operative CCDs that remained in communication with the HEU (Figure 8).
Figure 8: Event logger extract from locomotives 4420 and 4472, locomotive 4420 (HEU) brake control applications
Source ATSB
The CCDs, remote locomotives and EOTM beacon that could not detect the HEU beacon via the trainline cable subsequently broadcasted an exception message, which was received by the other devices along that portion of the trainline. As each device received more than one exception message within 5 seconds, all operative CCDs and remote locomotives self-initiated an emergency ECPB brake application to stop that part of the train. As this was a system-initiated application and only the trainline communication was interrupted, the brake pipe remained intact and fully charged (Figure 9).
Figure 9: Event logger extract from locomotives 4420 and 4472, locomotive 4472 (remote) and ore car CCD brake application
Source ATSB
Following the triggering of the 120% TBC application, the brake interlock feature held the brake application on the lead locomotives and CCDs in communication with the HEU. The interlock remained active, maintaining this brake application, as the driver had not reset the ECPB emergency condition.
The self-initiated emergency applications on the remote locomotives and associated ore car CCDs maintained brake application on the respective vehicles. Communication with the HEU was interrupted, so power and control commands (reset) were not available. Consequently, the CCDs relied on sufficient battery charge, and the 60-minute shut-down feature, to keep the brakes applied.
As indicated in Figure 8, the independent brake was fully applied at 0353.
For train M02712 the operative CCDs, remote locomotives and EOTM that were not in contact with the HEU beacon shut down about 60 minutes after the loss of power supplied by the trainline cable (that is, at about 0438). As the brake pipe remained charged, the air brakes released on these ore cars and the remote locomotives. The release of these air brakes and the incomplete application of handbrakes on train M02712 resulted in the train commencing to roll away at about 0440 (Figure 10).
Figure 10: Event logger extract locomotives 4420 and 4472, locomotive 4472 (remote) and ore car CCD brake release
Automatic train protection system
The 4 locomotives on train M02712 were each equipped with an Alstom Ultra-Cab II (UCII) microprocessor-controlled automatic train protection (ATP) system. The UCII system was not a standalone system; it interfaced electronically with other onboard equipment including the FIRE system, ECPB system, wayside transponders and other control systems that combined to provide for the safe operation of the train within the parameters defined in BHP’s rules and procedures.
The ATP functions included checking the locomotive speed and supervising its operation within the limits imposed for the track section. If the locomotive exceeded the target speed limit, alarms would sound to prompt the driver to reduce speed.
The locomotives carried a radio transmitter, transponder reader and antenna. The equipment relayed transmissions between the locomotive and transponders fastened to the track crossties (sleepers) at key locations, such as ATP entry and exit points and interlocked wayside signals along the railway. Track-mounted transponders relayed unique location identification and target speed data to the locomotive UCII microprocessor.
The driver had to reduce speed to the target limit within a predetermined time. If this did not occur, the ATP system automatically communicated with the braking system to request a brake application to stop the train. The type of brake application depended on the setup of the locomotive at the time of the command:
If the locomotive was configured for conventional pneumatic braking, the ATP triggered a service braking application. If this was ineffective in slowing the train, the ATP then triggered a penalty braking application.
If the locomotive was configured for ECPB, the ATP requested a penalty braking application only.
Additionally, when the locomotive was stationary with its reverser in the neutral (centre) position and the ATP detected a train movement of more than 0.5 m, the ATP requested a penalty braking application to prevent a potential locomotive runaway.
Each locomotive’s ATP system automatically configured to mirror the ECPB brake setup for that locomotive, either as a HEU, trail unit or remote unit. The ATP would not enforce target speed limits or runaway protection on locomotives configured as either a trail or remote unit.
The way the brake controller actioned the ATP’s request for a braking differed depending on whether the train was configured for conventional pneumatic mode or ECPB mode:
In conventional pneumatic mode, braking signals from the HEU brake controller were propagated to the remote locomotives and ore car pneumatic valves via a reduction in brake pipe pressure. A penalty brake request by the ATP system caused the brake controller to vent the brake pipe pressure to apply and hold the brake application on the locomotives and ore cars, until actioned by the driver.
In ECPB mode, braking signals from the HEU brake controller propagated to the remote locomotives and ore cars electrically via the trainline. The brake pipe remained charged with air. A penalty brake request by the ATP system caused the HEU brake controller to apply the brakes on the lead locomotives. The controller also transmitted an electric signal via the trainline to the remote locomotives and CCDs in each ore car to apply the brakes. An interlock feature in the brake controller then maintained the brake application until actioned by the driver.
In other words, the ATP did not directly integrate with the pneumatic braking system and could not operate a valve to dump brake pipe pressure. Accordingly, if a train was being operated in ECPB mode and there was an ECPB emergency/penalty braking application, and the train then commenced rolling away, subsequent penalty requests by the ATP system would be ineffective when the brake pipe remained charged.
BHP advised the ATSB that implementation of the ATP system in this configuration was historical and done to manage other operational issues such as derailment and ATP override.
The ATP in each of the 4 locomotives in train M02712 functioned respectively as a HEU (4420), trail unit (4434) and 2 remote units (4472 and 4440). When train M02712 started to roll away, and when later passing signals set to red or attaining an overspeed condition, the ATP system in the HEU triggered a penalty command (100% TBC) to the brake controller. In each instance, the ATP system penalty commands to the ECPB system were ineffective in stopping the train.
The brakes on the lead locomotives and the ore cars in communication with the HEU had already applied in response to the first 120% TBC (due to loss of trainline communications) and remained applied due to the brake interlock feature. Later ATP system calls to the EP-60 brake controller and responding ore cars had no material effect in mitigating the runaway of train M02712.
Vigilance control
The train’s vigilance control system checked for driver activity and automatically stopped the locomotive/train when there was no driver-initiated control input or response from the driver to aural and visual warnings displayed via the FIRE system. The vigilance system used random timing and task linking to check for driver activity.
The vigilance system was active when the locomotive air brake was set as the HEU and the locomotive air brake cylinder pressure was less than a predetermined level (independent brake released). The vigilance system was suppressed when any of the following occurred:
the locomotive air brake cylinder pressure was greater than a predetermined level (independent brake applied)
the locomotive’s braking system was set to trail or remote
the reverser was in the neutral (centre) position
the locomotive configuration was set for operation at a defined slow speed.
Prior to exiting the locomotive cab to apply handbrakes, the driver of M02712 applied the independent brake fully (increasing the air brake cylinder pressure above the predetermined level) and placed the reverser in the neutral (centre) position. These actions formed part of the ‘three-step process’ that the driver was to action when securing a locomotive (see Rules and procedures for securing trains before conducting work). The implementation of these actions had the effect of supressing the vigilance system, so the system had no effect during the runaway.
Risk management
Risk assessments for material risks
BHP’s Rail Safety Management Plan stated that the risk profile of BHP’s rail operations was determined by its risk management procedures. Different procedures were used for ‘material’ and ‘non-material’ risks. Material risks typically related to fatal accidents and significant operational or catastrophic events. Non-material risks related to task-based hazards and events with less severe consequences, and such risks were typically managed through BHP’s health, safety and environment (HSE) risk management processes.
BHP’s Risk Management Procedure provided guidance to the managers (risk owners) responsible for managing material risks in their area of responsibility. It outlined a series of phases, which included establishing the context, risk assessment (including risk identification, analysis and evaluation), risk treatment, and monitoring and review.
The procedure stated:
A formal risk assessment is a team-based, risk assessment process which provides an efficient and effective method of risk identification, risk analysis, assigning controls and developing risk remediation plans...
It involved relevant subject matter experts and the potential control owners identifying and agreeing on (material) risk events, and then:
For each risk event, identify potential causes and impacts. When assessing risk, normal operating conditions, abnormal operating conditions, start-up and shutdown activities and potential emergency situations shall be considered…
Based on the causes and consequences, the team must identify credible controls that will prevent, detect or mitigate the risk event and associated causes and consequences. The controls identified must be based on the hierarchy of controls[17]… The controls will be either preventive or mitigating controls.
A bowtie was used as the data capture tool for material risk controls. The procedure stated that:
…A bowtie is developed in a workshop with the relevant subject matter experts, risk owner and potential control owners.
Once the risk event, causes and impacts have been agreed, the critical controls are identified. A critical control is a control that significantly reduces the likelihood and / or impact of a material risk. The number of risk controls must be appropriate to the risk event and must play a key role in achieving the business objective…
A material risk required a control design assessment (CDA) occur to ensure the critical control were suitable following their creation and when changed. The assessment test varied dependent on whether the critical control was a procedural (administrative) or an engineering solution. Additionally, critical controls underwent a unique control effectiveness test (CET) to give assurance that each control was in place and effective in managing the material risk to an acceptable level.
In other words, the identification of a material risk event supplied a method for focusing management oversight on the critical controls preventing or mitigating the risk from such events.
Rail-mounted equipment interaction incident
BHP had identified a material risk event titled ‘Rail Mounted Equipment (RME)[18]Interaction Incident’, which referred to events involving an uncontrolled interaction between RME and people and RME and RME. The risk assessment was initially developed circa 2013, with numerous changes made in July 2016.
A range of scenarios were included, which considered the likelihood and consequence of interactions between RME/RME, RME/road vehicles and RME/track worker(s) resulting in the potential for single or multiple fatalities. The maximum foreseeable loss scenario involved a hi-rail vehicle (not covered by ATP) striking maintenance workers. Other worst plausible scenarios included shunting activities, driver walking around the locomotive at night, track workers working on an adjacent line, workers within the 3 m zone.
The bowtie identified the following 12 ‘causes’ that could lead to such an event:
at risk behaviour of personnel, working outside rules/procedures/instructions
failure of or poor communications, radio protocols not followed or equipment failure
ATP system failure or overridden
ineffective track protection applied
limit of authority terminals / operator error for road-rail vehicles or track machines
uncontrolled or uncommanded movement of RME (workshops and main line)
exceed limit of authority (or no authority)
non-compliant track design
signal system ineffective
ineffective train control management
operator ignores derailer at active car dumper or train load out
derailment resulting in fouling of adjacent line.
The uncontrolled or uncommanded cause included the following types or examples:
brake isolation / failure
failure of tower control in non-safe state (J-Hub facility)
malicious damage (vandalism)
failure to secure vehicle against movement (braking and chocks).
It is not clear if the ‘failure to secure vehicle against movement (braking and chocks)’ related only to RME/trackworker(s) situations where maintenance personnel required access to work on rolling stock, or it was also intended to include securing a train against a runaway event on the main line involving a service train.
A matrix within the bowtie then linked each of the 12 causes to one or more preventative critical risk controls. The following 7 critical preventative risk controls were identified for an RME interaction incident caused by the uncontrolled or uncommanded RME movement:
radio communication - the protocols to enable clear verbal communication between train control and/or workgroups to confirm and acknowledge authorities to proceed with the related activity
rolling stock (excluding ore cars) maintenance - the asset management plans for RME to prevent derailment from equipment failure
signalling systems - the systems and procedures to positively locate and provide safe separation between RME
isolation protection - the procedures for the isolation and protection of RME against unintended movement (hand brakes and roll away protection)
trained and competent - requirement for the engineering and operational personnel to have the correct competencies for the rail-related tasks being carried out
three-step protection - the rules and procedures applicable to all personnel entering the profile of an RME to protect against the unauthorised movement of rolling stock
interface coordination - the agreements to clearly delineate the responsibilities of each party or functional area to facilitate the interaction between those parties and rail operations at each interface point.
The mitigating (or recovery) control listed for this cause was ‘corporate affairs and legal support’.
The preventative controls that were identified within the bowtie for other causes also included the ATP system. The documented aim of the ATP system control was to ensure trains did not exceed permitted speeds and/or levels of authority (that is, passing a signal at stop). The ATP system was stated as meeting the objective through monitoring target speeds and location information to provide warning to the driver if they were likely to exceed a defined speed profile or a braking curve profile for a limit of authority. The control specified that the ATP system would apply the brakes (penalty) if the driver did not respond to the warnings.
The matrix within the bowtie linked ATP as a preventative control measure to the causes on:
- ATP system failure or overridden - poor maintenance of onboard or way side system
- Limit of authority (LOA) terminals - failure or operator error for road rails vehicles/or trackside machines; GPS failure in non ATP RME (equipped with back box) or not displaying correct location
- Non-compliant track design - construction, renewals and/or maintenance, RME operating outside of safe envelope (out of gauge), incorrect placement of Insulated rail joints (or misalignment to ATP map), parking of RME beyond fouling points, out of gauge vehicle
The bowtie risk assessment did not link ATP as a preventative control against the uncontrolled or uncommanded movement of RME and it did not link ATP as a mitigating control for any of the causes of an RME interaction incident.
Evaluation of critical controls
Each critical control listed in a bow tie linked to various design and operating standards, together with the criteria used to verify the control’s overall effectiveness. The design and operating standards for the controls relevant to securing a vehicle (train) against an uncontrolled or uncommanded movement linked to the content of relevant modules within the rail rule book, procedures, worker competency programs, asset management plans, interface coordination plans and emergency response plans. Their effective implementation was dependent on the qualified workers undertaking the related task described within the documents.
The scope and design standards for each critical control primarily addressed material risk associated with personnel accessing the profile of rolling stock to undertake maintenance or recovery tasks. Apart from a reference to drivers keeping an awareness of current operating instructions, the critical controls contained no reference to the risk of a significant or catastrophic event arising from an RME main line runaway, such as M02712.
The methods used in a critical control verification (CCV) included the scheduled inspection of records, auditing and observation of behaviours targeting representative samples of the workforce. Results from the CCVs and review of the CDA additionally fed into a test plan used to review periodically the significance of key changes to associated procedures, standards and permits or those triggered by significant events or audit findings since the last CDA and CET if applicable.
The outcome from the review processes (CCV, CDA and CET) formed the material risk control assessment (MRCA), where a rating was assigned to the material risk event’s critical controls of either ‘well controlled’, ‘requires some improvement’, ‘requires significant improvement’ or uncontrolled’.
The MRCA of the RME interaction incident risk event undertaken in September 2017 found that, although the majority of critical controls were ‘acceptable’, the risk of an RME interaction with people rated overall as ‘requires some improvement’. The bowtie version date recorded another review and update occurred on the 14 February 2018. Records of tracked changes for the MRCA’s indicated changes occurred in the bowtie on 16 November 2017 and 8 January 2018 respectively to the preventative controls trained and competent and rolling stock [excluding ore cars] maintenance. There was no record of the changes to the bowtie associated with the update conducted on the 14 February 2018.
CCD shutdown event in March 2017
On 27 March 2017, the driver of a loaded ore train reported an ECPB 120% penalty brake application that occurred at Garden (205 km). The driver reported the condition of ‘EOT off’, ‘power off’ and ‘communication loss’ to remote locomotives. The driver also reported placing the automatic brake handle in the full ECPB service position before disembarking to apply handbrakes to the train. The brake pipe remained charged at 600 kPa.
On returning to the locomotive cab, the driver reported noticing a ‘?’ symbol displayed for the remote locomotives on the FIRE screen. After consulting with the maintenance centre staff about the symbol, the driver reverted the configuration of the train from ECPB to pneumatic operation and then continued the journey to Port Hedland without further incident.
On the 28 March 2017, BHP maintenance centre staff conducted tests to repeat the conditions of the reported event while watching the ECPB system response. The testing found that, after a disconnection of the trainline cable interrupting power to the CCDs and EOTM, the ECPB application released when the CCD batteries started to fail. They also noted that they now required drivers to ‘dump their trains’ (vent train brake pipe to atmosphere) in the case that there was a loss of trainline communications and there was no EOTM brake pipe reading.
BHP’s enquiries with the ECPB vendor (NYAB) later that day confirmed that, after a break in the trainline cable, the trainline power would shut down for the entire train. In addition, the vendor advised:
Ore car CCDs to the rear of the point of break, with brake pipe charged and no HEU beacon detected, would cut out and shut down after 60 minutes.
Ore car CCDs in front of the point of break, with brake pipe charged and HEU beacon detected, would maintain ECPB brake application until the internal battery charge depleted. The CCDs would then release the brake application, regardless of the detection of the HEU beacon.
The vendor agreed with BHP staff that the above conditions meant that dumping the brake pipe pressure through a driver-initiated application of the automatic brake handle to the pneumatic emergency position, together with applying handbrakes (the number dependent on track grade), was necessary to avoid the possibility of a train roll away.
The operating instruction containing procedures for responding brake pipe emergencies was amended on 5 April 2017 (see Operating instruction 17-11). No addition was made to the RME interaction incident risk assessment to include a brake pipe emergency as a cause of an uncommanded train movement, and the procedure for responding to a brake pipe emergency was not included as a critical control. In addition, the effectiveness of the control was not examined by an MRCA or related processes.
Management of change
BHP management of change process
BHP’s Rail Safety Management Plan stated:
BHPIO Rail employs a management of change process which deals with permanent, temporary or incremental changes, to organisation, plant, equipment, materials, standards or procedures, and changes associated with laws and regulations in relation to BHPIO rail activities.
The process is used to manage the changes which may have a health and safety impact to identify and control potential risks associated with the change. Activities requiring change management are identified via the BHPIO Management of Change procedure where actions are assigned to the relevant departments and managed in 1SAP.[19]
BHP’s Management of Change Procedure stated the overall purpose of the procedure was:
… define the process for managing risk associated with change by ensuring the appropriate process steps are followed and recorded.
Change could be an engineering or non-engineering change. This includes alterations to plant, infrastructure, equipment, products, materials, process systems, management systems, standards, procedures, removal or introduction of people or a change to the environment.
The scope of the procedure included changes of either a permanent, temporary, or emergency nature, or where the untreated risk score exceeded a defined value.
The 5 steps forming the management of change process comprised of:
initiate / design - document the reason / basis for the change, engage stakeholders and prepare the proposed change with sufficient detail for the formal review process (including identify subject matter experts, conduct a risk assessment, identify impacted areas and associated actions to manage the risk)
review - assess and accept risks and controls associated with the proposed change
approve - approve risk assessments, check the right people and reviewers have been engaged and accept accountability for the change
implement - confirm / validate the controls to manage risk are in place
closeout - verify changes implemented.
BHP’s introduction of the ECPB system to main line operations
On the 3 June 2011, BHP Iron Ore started a management of change process for the introduction of the ECPB system to main line operations, including the Yarrie line.[20] The change was part of a broader project to increase rail capacity.
The management of change assessment presented a series of questions guiding the change originator through the process steps and various topics for consideration. Fields enabled the recording of relevant responses, comments, supporting documentation and details of the risk ranking before and after the change. The risk assessment linked to the HSE risk management procedure used to assess non-material risk related to task-based hazards and events. The assessment had no entry detailing the risk scenarios assessed or their resultant ranking.
The assessment listed supporting documentation that recorded processes undertaken by BHP when introducing ECPB. The documentation included risk assessments, change plans and communication plans. BHP was unable to retrieve and supply any of the listed documents for review by the ATSB.
In April 2012, BHP engaged an external provider to undertake a study[21]verifying the benefits of converting the existing fleet of conventional pneumatically-braked rolling stock to ECPB operation, and the potential for increasing train length. The study utilised instrumented ore cars in rakes with both ECPB and standard pneumatic brake capability and compared the braking performance of each configuration with an emphasis on coupler forces and stopping distances. The study identified key observations related to in train forces, stopping distances and section running times, concluding that although some problems and delays associated with brake equipment failure occurred during the study, the actual performance of the ECP brakes was assessed as very good.
In December 2013, BHP completed a report on the selection phase, which built on the previous work that investigated avenues to increase rail capacity through initiatives such as operating longer and heavier trains. The report identified that the BHP fleet now operated a mixture of ECPB capable and non-ECPB capable locomotives and ore cars. The report quantified the scope of rolling stock that needed to be converted together with a project proposal for the conversion of the ore-cars and locomotives as part of the ECPB project, in order to complete the transition and commence ECPB operation across the fleet.
The report noted that operational trials conducted in 2013 placed greater emphasis on quantifying the benefits of ECPB in terms of cost savings, mitigation of production losses and sectional cycle time improvements. The report then detailed further work required, which included:
retrofit all non-ECPB ore cars and locomotives with ECPB
train and certify all locomotive drivers in ECPB
ATP analysis and updates, focusing on amended braking distances for ECPB-equipped trains.
signalling updates (if required) based on revised braking curve analysis.
At that stage it was intended to complete the training of drivers and commissioning of the ore cars and locomotives by mid 2015.
In May 2014, BHP completed a report on the definition phase of the project. The scope included the retro fitment and/or commissioning of ECPB to all mainline rolling stock with the following specific objectives:
retrofit ECPB equipment to locomotives and ore cars
train locomotive drivers in the operation of ECPB rolling stock
train rail workshop and yard maintenance personnel in the repair, replacement and troubleshooting of ECPB and ECPB related systems
commission existing ECPB locomotives and ore cars
update network ATP maps to ensure safe operating conditions in operating 40 t axle loads
upgrade relevant train-load outs to high accuracy track scales to reduce loading variability to prevent exceedance of bridge overloading parameters at 40 t axle loads.
The report included detailed explanation of the planning and processes implemented to modify the rolling stock for installation of the ECPB equipment and the arrangements to ensure the reliability and integrity of the ECPB system within the BHP rail network.
To facilitate the latter, BHP proposed the ECPB operations team design a validation program in consultation with the rail operations group considering:
identification of potential issues (including starting testing and validation activities in the definition phase rather than wait until the commissioning period)
industry knowledge and learnings from other sources who were using ECPB
operational engagement.
The validation scope also documented the testing of a range of functions, including:
emergency and penalty recovery – testing procedures using the ECPB interlock feature to recover trains (as existing recovery procedures only related to conventional pneumatic braked trains)
securing of trains – following the failure of either the pneumatic or ECPB systems, securing trains by the application of handbrakes (with procedures to be reviewed in conjunction with the ECPB interlock functionality trials)
ECPB interlock – using the interlock to secure the train instead of applying handbrakes
handbrake tables – undertaking static and dynamic handbrake trials to demonstrate the potential of reducing the amount and times that handbrakes needed to be applied and updating the handbrake chart accordingly.
With respect to the identification of potential issues, the definition phase report noted that twice during the main line trials, it was observed that a percentage of operable brakes in a train decreased for no apparent reason. This observation was reported to the supplier, and it was concluded that the low percent of operable CCDs was due to the large number of ore cars with low batteries. The events related to situations where the driver turned off the trainline power, but left the ECPB active, forcing the CCDs to run on battery power to maintain the ECP brake application. It was observed that after around 11 hours, the CCDs gradually started to cut out due to low battery. The system was left in this state for about 23 hours before the operator turned the trainline power back on. This left all CCDs in the train with depleted batteries.
The conditions present in the train for the above 2 events differed from those observed in the March 2017 event (that is, the CCDs maintained communication with the HEU). However, these events demonstrated that, in the absence of trainline power, the CCDs battery charge could be depleted to a level where the CCD would cut out and either not apply or maintain an ECP brake application. As a result of these events during trials, BHP required that, for main line operations, the number of operable brakes in an ECPB-configured train must be above 93%. If this level could not be achieved, the driver was to revert the train to conventional pneumatic braking, after informing train control.
The report summarised other actions undertaken by BHP, including risk assessments for the transition to 40 t axle loads and the ECPB conversion process for the introduction of longer and heavier trains, and an FMEA for the transition to 40 t axle loads. BHP was unable to retrieve and supply these documents for review by the ATSB.
Overall, the definition phase report noted that the project was expected to be completed in the third quarter of 2015.
Systems engineering processes
Risk management and management of change are closely-related functions and processes within a safety management system, and they both interact with many other organisational processes. When developing and implementing a new physical system or product, there are also a range of engineering processes that can be applied.
Systems engineering is a way to manage complexity in the development of a product or engineered system. It is a concept that began to form in the 1940s and was later given impetus by the United States National Aeronautics and Space Administration as well as the United States military as a way to manage system complexity through control of the development process (Leveson, 2016). The process can be applied to the development of any product.
As noted by Kusumo (2019):
Given the increasing complexity of railway systems, the integration of a system with new or legacy systems may cause unintended behaviour of any of the systems, as well as the whole integrated system. As such, the traditional approach to delivering major rail projects where each railway system is designed, implemented and tested in isolation of other interconnected systems is no longer sufficient to ensure that the rail network can be operated safely.
A systems approach to implementing change in the railways, especially in integrating multiple new or legacy rail systems is required to ensure that the railway’s operations remain safe, so far as is reasonably practicable. This approach needs to cover the system life cycle: including requirements analysis, system design, system implementation, testing and commissioning, service operation and maintenance, as well as decommissioning.
The V-model is commonly used to describe the systems engineering process (for example, see Figure 11). In this model, the product’s user defines a concept of operations or broad set of requirements that describe what the product should do. This description is broken down into a progression of increasingly detailed sets of sub-requirements. The higher-level requirements describe all aspects of what a product should do and how well it should do them, and the lower-level requirements describe the product’s architecture and other detailed elements of design. The requirements drive the design; that is, design elements are chosen based on the best way to meet the requirements. The designers also need to devise ways to prove that the requirements have been met, which forces them to make the requirements verifiable. Safety can be, and often is, among the design objectives.
Figure 11: Systems engineering lifecycle V-model
Source: Systems Safety Assurance Guideline – RISSB, 18 September 2018
Once a design is finalised, the design and product passes through increasingly broader sets of verification and validation activities (such as testing) to show that the design does meet the requirements. Verification is the process of showing the final product meets the requirements (that is, the product was built correctly). Validation is the process of showing that the requirements met the user’s needs (that is, the right product was built). If the requirements definition and design process are conducted well, there should be few or no ‘unpleasant surprises’ throughout the verification and validation process.
System integration refers to the progressive assembling of subsystems so that the broader system, as an integrated whole, is able to deliver the overarching functionality. A key component of system integration is defining system interfaces and assessing identified hazards associated with those interfaces. More specifically, as stated by Kusumu (2019):
To ensure safe integration, the SRS [system requirements specification] for a system needs to consider the interface requirements between it and any subsystems and between it and any existing or legacy systems. In addition, these interface requirements for the new system need to include any Safety Related Application Conditions (SRACs) on the existing railway systems that will impact the new system…
A system approach to designing railway systems that will ensure safe system integration involves a systematic analysis of the following:
Interface compatibility between connected railways systems (data, power and signal, etc.);
Risks associated with failures of interface between interconnected systems;
Risks of system failure which may compromise the overall safety of the railway operations;
Compliance with …SRACs from any existing or legacy systems; and
Verification that the identified risk controls have been incorporated in the system design.
Hazard identification and risk analysis is an ongoing and iterative process that is relevant to multiple phases of the lifecycle model. A range of safety assessment techniques can be used to identify problems with the product design, such as interface hazard analysis, functional hazard assessment, fault tree analysis and failure mode effects analysis (FMEA).
A wide variety of standards and guidance documents on systems engineering processes have been published in recent decades, both generic and tailored for specific industries. In 2013, system safety guidance material specific to the rail industry was collated into an International Engineering Safety Management Handbook,[22] which was aimed at ‘clearly outlining the activities involved in making a system or product safe and providing the evidence that it is safe.’ The extent to which this and other guidance reached the Australian rail industry is unclear. A paper published in February 2021 concluded that the Australian transport sector lacked national direction in the application of systems engineering compared to the defence sector or that provided in other countries (Welschen and others 2021). Nevertheless, the application of systems engineering principles has gradually increased through ONRSR safety messaging and publication of guidance material by RISSB.
BHP application of system engineering processes
BHP’s internal investigation into the runaway and derailment of M02712 included a range of internal rail operations and engineering specialists. The investigation report defined a ‘systems engineering framework’ as:
A structured engineering process which applies, in alignment with ISO 15288,[23] a systems engineering approach to the management of risks associated with the introduction of changes to complex systems over their life cycle.
The report concluded that BHP’s WAIO rail network did not have a systems engineering framework in place for the introduction of ECPB into its existing braking systems. It also noted that:
there was no formal system assurance process in place to identify and address safety-related matters that were an outcome of system integration activities.
when ECPB was introduced and modifications were made to the existing ATP system, they were largely managed on an individual system level, rather than giving full consideration to the aggregate function required to be performed by the braking system as a whole
there was insufficient focus on system integration in the risk assessment phase of the ECPB project, particularly on critical controls that were reliant on system integration for safety functions (such as the ability of ATP to effectively intervene).
In 2013, the vendor modifying the ATP system software to facilitate yard auto mode of a loaded consist, controlled by the dumper automated spotting of locomotives (DASL) system, identified 13 safety-related application conditions (SRACs).[24]The vendor communicated to BHP that the conditions listed were outside the scope of work and therefore required BHP to evaluate and mitigate the conditions to a risk level acceptable to BHP. The SRACs included several system conditions that, if present, could result in an unintended train movement that could result in derailment, collision, or runaway event.
Another vendor undertaking later modifications to the ATP software in 2015 and 2016 again raised the SRACs with BHP.
BHP provided no record to the ATSB of its review of the information provided by the vendors or its assessment of the identified risk associated with each SRAC, or of subsequent changes to procedures or systems to address the safety-related information provided by the vendors. There was no indication in the BHP risk assessment for an RME interaction incident risk event that information from the SRACs was included in consideration of reasons for (or controls for) an unintended train movement. In addition, BHP’s internal investigation into the runaway and derailment of M02712 found that there was no formal system assurance process for addressing safety-related information provided by vendors (including SRACs).
Trial trainline inter-car connectors
A trainline interconnection between rail vehicles (that is, locomotives or ore cars) was via an inter-car cable assembly. The assemblies terminated at a junction box located at the end of each rail vehicle and were joined using a connector. BHP’s ore trains used polarised NYAB Freight Mate connectors (also known as Tri Star connectors) (Figure 12).
Figure 12: NYAB Freight Mate inter-car connector
Image showing typical plug-type device connecting the trainline cable between ore cars.
Source: BHP, annotated by the ATSB
In mid-2018, BHP initiated a trial of a Wabtec type connector (Figure 13) to address repeated service delays associated with trains receiving a 120% TBC brake application due to a loss of trainline power and communications from continuity faults within the existing connector. Associated with the service delays, BHP also noted that, depending on the track location, the failure required the application of handbrakes to secure the train. Undertaking this task increased exposure of the train crew and support personnel to a potential injury.
BHP managed the trial through the implementation of its management of change (MoC) process and supplied advice of the change to the ONRSR through a notification of change to railway operations (associated with a ‘change to a safety critical element of existing rolling stock’). The proposed commencement date was 12 September 2018.
The risk assessment undertaken in conjunction with the MoC process primarily targeted the identification and management of work health and safety considerations that could arise during the initial installation or follow-up monitoring work on the connectors. In addition, one operational risk related to the failure of a Wabtec connector causing a communication loss through the trainline cable. The consequence assessed was a potential financial loss due to service disruptions of main line operations when responding to and recovering from the fault.
The trial involved installing 12 Wabtec inter-car connectors to a combination of 7 recently overhauled Brakden QRRS and Golyns type ore cars. Train M02712 was the only train fitted for the trial, with the 7 ore cars found from positions 2 to 8 inclusive in the first rake.
Rail operations personnel received an operations notice that the trial would begin on 1 November 2018. During the 3-month trial period, BHP planned to check the performance of the inter-car connector at 2-weekly intervals to find potential faults related to the performance of the connector. BHP stipulated criteria whereby any failures related to a loss of EOT communications resulting from the assembly failing, connector pulling apart or premature wear under normal operations would result in the immediate termination of the trial and reinstatement of the NYAB inter-car connectors.
The driver of M02712 recalled that the loss of trainline communications occurred as the train was passing over a level crossing. They also stated that, while applying handbrakes to the ore cars, they saw a detached or disconnected inter-car connector. The driver recalled the position was at about ore car 10 in the first rake and the connector did not appear damaged, just disconnected with both ends hanging down. The driver recounted that it was a larger new type of twist connector of a type that the driver was not familiar, rather than the rectangular type of connector with which they were familiar.
During the runaway occurrence involving M02712, dragging equipment detectors placed on the track next to signals GNN3, 202W and 199W recorded strikes from dragging equipment. It is likely the strikes were from the trailing ends of the disconnected inter-car connector.
BHP recovered only one of the 12 trial Wabtec type inter-car connectors from the wreckage of train M02712. The exact position where the break in trainline communications occurred or the location of the recovered connector in the trial ore cars was unknown.
The management of change records confirm that on the 28 December 2018, BHP ceased the trial of the Wabtec connectors because of the runaway of train M02712 on 5 November 2018. BHP cited the reason as:
Trial was aborted due to an alleged cable striking a crossing and having an emergency brake application. When the cables were installed the car was empty however when loaded the cable may have been hanging low. Trial will have to be repeated with checks conducted to ensure cables on all types of ore cars are not hanging too low allowing them to make contact with infrastructure.
Figure 13: Wabtec inter-car connector
Image showing the trial plug type device connecting the trainline cable between ore cars.
Source: BHP, annotated by the ATSB
Rules and procedures for brake pipe emergencies and penalties
Overview of manuals and instructions
The BHP Rail Rule Book included 16 modules outlining procedures related to rail operations. BHP sent a revised version of the rule book to the Western Australia Office for Rail Safety[25] (ORS) for endorsement on 15 April 2013. References to ECPB operations were not included at that time, as BHP did not anticipate introducing ECPB into service in the current hard-wired format prior to the release of the revised rule book.
Delays from the ORS in assessing the rule book resulted in the final endorsement not occurring until 22 August 2014. Following receipt of the ORS endorsement, BHP developed training and awareness packages, publishing the rule book ‘as endorsed’ in February 2015.
During the endorsement period, BHP also began arrangements to modify the ore train fleet from pneumatic braking to an ECPB overlay system. Although the recently published rules and procedures addressed conventional pneumatic operation only, BHP managed the changes to the operational rules for the implementation of ECPB through the issue of a series of operating instructions to rail workers.
The rule book Module 1: General rail rules and procedures, sections R1-1.2 and R1-1.3, defined the compliance responsibilities of all workers/persons in relation to the rules and procedures as:
R1-1.2 Application of rules
The compliance with the BHP Billiton Iron Ore Rail rules and procedures is mandatory for:
- all workers engaged in the operation of the rail;
- all workers working within the Danger Zone; and
- all persons entering on to the rail premises other than any areas to which public have unrestricted access.
All persons to whom these rules and procedures apply are responsible for ensuring that they remain familiar with these rules and procedures, including any amendments issued.
R1-1.3 Operating Instructions, Notices and General Alerts
All persons to whom these rules and procedures apply shall familiarise themselves with all current applicable Operating Instructions, Operating Notices, Safety Notices and General Alerts upon commencing duty.
BHP confirmed these statements meant an issued operating instruction was effectively an update/amendment to the applicable rule. All persons associated with rail operations were then responsible for ensuring they remained familiar with the rules and procedures, including any amendments (operating instructions) issued.
Process for issuing and receiving operating instructions
BHP’s procedure 0119630 (Issuing and receiving operating instructions) stated:
Operating Instructions are a notification to rail personnel of relevant information about changes to safeworking procedures, work instructions and safeworking infrastructure changes which alter their work duties and responsibilities. They are not a means of facilitating operational or temporary change that would in the daily course of business be distributed by the issuing of memorandums from the various supervisors or line managers within BHP Iron Ore Rail.
The procedure stipulated that the following processes were applicable for issuing an operating instruction related to changes to safeworking procedures, work instructions or the rule book:
- review session involving key stakeholders / representatives affected by proposed changes are to be held, minutes taken and documented attendance records completed;
- a risk assessment by representatives of key stakeholders impacted by the change is to be conducted and incorporated as part of the Change Management Process;
- Change Management Process is to be applied in line with BHP Iron Ore Change Management Procedure…;
- Rail Operations Safeworking team is to be supplied with a signed copy of the Change Management and Risk Assessment; and
- an information package for all complex changes to rail rules, work instructions and safe working procedures to be presented to relevant stakeholders prior to implementation.
BHP confirmed that, although a new rule implemented via an operating instruction should be subject to the processes detailed above, the adoption of the processes could vary when an operating instruction was re-issued following minor changes, such as rewording based on feedback from end users or when updated. This practice was accepted so long as the intent of the rule remained unchanged. When a change or amendment occurred to a rule contained in the rule book, the procedure also triggered the submission of a notification of change to the rail safety regulator.
An amendment to alter an operating instruction related to the rule book did not have an expiry date or period specified where it could remain in circulation before incorporation into the relevant module of the rule book. Instead, an amendment required the issue of a new operating instruction, which rescinded the original instruction. This was in contrast to instructions unrelated to the rule book, which required a review to occur before the end of a 6-month period to decide if the content was still applicable and required on an ongoing basis. If considered required, the operating instruction would transition into an amendment to the rule book or a work instruction document.
However, BHP confirmed there was a program underway to review published operating instructions and incorporate relevant changes into the associated modules in the rule book. The safety specialists undertaking the review prioritised their work by focusing on the modules that had the most operating instructions published. At the time of the runaway of M02712, safety specialists had reissued 11 modules of the rule book. The review of module 6, relevant to brake pipe emergencies and penalties, had not begun at that time.
There was no requirement or guidance in the procedure for issuing operating instructions to require that the reason for a change (or consequences of not following the amended procedure) be provided. In addition, there was no stated requirement for new or important information to be presented in any particular format (see also Formatting of rules and instructions). In contrast, the equivalent procedure for issuing operating notices required that when replacing an operating notice the content changes were to be ‘highlighted’.
BHP publicised changes to operating instructions through safe start rail briefings conducted for drivers each day. Operating instructions and other notices were also available on the BHP intranet for review by drivers.
Rule book procedures for responding to brake pipe emergencies and penalties
The rule book Module 6: Rail operations, section R6-3.0, supplied instructions to drivers responding to brake pipe emergencies and penalties[26]on trains with conventional pneumatic braking systems. It stated:
When an emergency brake application (dump) has occurred either initiated or uninitiated by the driver:
a) Duplicated lines [such as at Garden]
Driver
- carry out emergency radio procedures;
- check if adjacent track/s is fouled; and
- where necessary, provide train protection.
b) Single line
- advise the train controller.
For calculating the number of handbrakes to be applied to a train, the train controller shall utilise the Handbrake Calculator (electronic)…
Section P6-4.0 supplied added procedural instruction for drivers to follow dependent on the conditions present when an emergency pneumatic brake application occurred. More specifically, separate instructions were provided for:
emergency brake application when train moving initiated by the driver
emergency brake application when train moving not initiated by the driver
emergency brake application when at a stand (not moving)
penalty brake application on empty train travelling more than 5 km/h
penalty brake application on loaded train traversing specific locations
penalty brake application on loaded train traversing other locations.
For the condition where the driver did not initiate the emergency application on a moving train, the procedures stated:
Driver:
- carry out emergency radio procedures (duplicated lines or where deemed necessary);
- advise train control;
- check if adjacent track/s is fouled;
- where necessary, provide train protection;
- employ train walk procedures during inspections;
- secure all portions of the train by application of the independent brake and handbrakes (as per handbrake chart);
- check for the cause of the air loss, or other indications;
- maintain radio contact with the train controller during inspections...
The procedures did not refer to the application of the automatic brake. For trains with a conventional pneumatic braking system, an uninitiated emergency brake application (brake pipe dump), whether initiated by the ATP system or a break in the brake pipe continuity, reduced the brake pipe pressure to atmosphere level, subsequently applying full braking effort to the locomotives (unless bailed off) and the trailing ore cars. If the brake pipe remained discharged, air pressure in the auxiliary reservoir on each ore car kept the brake application on that ore car while the reservoir kept sufficient pressure.
In other words, for a train with a pneumatic braking system, with the brake pipe discharged, the driver did not need to place the automatic brake handle to the emergency position to keep an emergency brake application.
With the train held by the emergency brake application, the rule book procedure required drivers to secure all portions of the train by fully applying the locomotive independent brake and the handbrakes on the ore cars[27]in accordance with instructions from the train controller (or in accordance with the handbrake chart if the driver could not contact the train controller). For the Garden location, where the track grade was around -1.5%, a loaded train with locomotive independent brakes applied required the application of the handbrakes to 100% of the ore cars.
Although the procedures referred to the application of handbrakes to secure the train, they did not refer to the associated three-step protection process (see Rules and procedures for securing trains before conducting work).
Operating instructions related to ECPB brake pipe emergencies and penalties
To facilitate the trial and introduction into service of the ECPB overlay system, BHP issued a series of operating instructions to supplement or modify the requirements of module 6 of the rail rule book related to brake pipe emergencies and penalties (Table 2).
Table 2: List of operating instructions related to brake pipe emergencies and penalties
Number
Title / purpose
Effective date
OI 14-14
Brake pipe dumps and penalties
21 February 2014
OI 14-18
Brake pipe dumps and penalties
14 March 2014
OI 15-35
Trial recovery process for ECPB trains after brake pipe dump or penalty
29 July 2015
OI 15-41
Brake applications – Mainline loaded trains
3 September 2015
OI 15-42
Reverting from ECPB to conventional pneumatic mode
4 September 2015
OI 15-43
Penalty brake application – Empty train
7 September 2015
OI 15-45
Penalty brake application – Empty train
8 September 2015
OI 15-46
Penalty brake application – Empty train
9 September 2015
OI 15-49
ECPB recovery process
30 September 2015
OI 15-53
Trial recovery process for ECPB trains after brake pipe dump or penalty
4 November 2015
OI 16-16
Brake pipe emergencies and penalties
17 March 2016
OI 17-09
Brake pipe emergencies and penalties
28 February 2017
OI 17-11
Brake pipe emergencies and penalties
5 April 2017
OI 18-72
Brake pipe emergencies and penalties
3 November 2018
Early versions of the operating instruction related to trial recovery processes following a brake pipe dump or penalty on ECPB trains. The trial categorised trains into the following 2 groups:
- Trains operating as conventional pneumatic only trains or trains operating in ECPB mode with EOT brake pipe displaying ‘off’
- Trains Operating in ECPB mode with EOT brake pipe displaying 0 kpa or above.
For ECPB trains with EOT brake pipe displaying ‘off’, the response from drivers to an uninitiated emergency brake application (referred to as ‘dump’) remained as per a train with a conventional pneumatic braking system (that is, rule book sections R6-3.0 and P6-4.0). The early versions of the operating instruction did not differentiate the required driver response between an emergency resulting in a dump of brake pipe pressure and the ECPB emergency brake application with EOT displaying ‘off’, where the brake pipe remained charged.
For trains in the second category, drivers were able to secure the train against unintended movement using the ECPB interlock feature.[28] In both cases, the driver had access to an ECPB support team via radio to assist in stepping the driver through the recovery process.
Operating instruction 16-16
The publishing of operating instruction (OI) 16-16, dated 17 March 2016, rescinded a series of earlier instructions related to brake pipe emergencies and penalties and replaced, in total, section P6-4.0 of the rule book. Although the operating instruction replaced a module in the rule book, BHP were unable to retrieve records of the consideration or implementation of procedures for issuing and receiving operating instructions, or a notification of change to the ONRSR.
In contrast to the rule book, procedures were now organised with separate instructions provided for:
emergency brake applications – ECPB trains
when train moving or not initiated by the driver with EOT displaying ‘off’
when train moving or not initiated by the driver with EOT displaying ‘0’ or above
when train at a stand initiated by the driver with EOT displaying ‘0’ or above
loaded train and EOTM displaying ‘0’ or above within specific locations
loaded train and EOTM displaying ‘off’
penalty brake application – pneumatic trains
empty train travelling less than 50 km/h
empty train travelling 50 km/h or more
loaded train within specific locations
loaded train in other locations.
Requirements in the instruction for drivers to secure an ECPB train following an emergency application with EOT brake pipe displaying ‘off’ were the same as previously for a conventional pneumatic train; that is, they still required the driver to:
- secure all portions of the train by application of the independent brake and handbrakes (as per handbrake chart)
OI 16-16 removed reference to the ECPB support team but included additional instruction to the driver on the procedure to recover from the emergency brake application. If the driver was unable to rectify the ECPB problem, these additional processes required a driver to place the automatic brake handle in the emergency position and reduce the brake pipe pressure to zero before conditioning the train for pneumatic operation.
A note at the end the procedure specified that drivers could not use the ECPB interlock to secure any train when reverting from ECPB to conventional pneumatic operation.
OI 16-16 included an expiry date of 17 September 2016. The reason for inclusion of an expiry date on an operating instruction related to a rule change was unknown and the instruction remained applicable until the issue of the next instruction on 28 February 2017.
Operating instruction 17-09
On 28 February 2017, BHP published OI 17-19, rescinding OI 16-16. To prepare for the changes, BHP implemented the processes detailed in the procedure for issuing and receiving operating instructions. BHP developed a management of change proposal that encompassed evidence of stakeholder consultation, risk assessments and testing of train dynamics under penalty and emergency braking conditions, and the proposal was submitted to ONRSR.
The objective of the change was to reduce complexity of the processes contained within the instruction, and the mitigation of work health and safety risk to drivers arising from exposure to various environmental hazards while undertaking a walk/inspection of their train. More specifically, the changes removed the requirement to inspect (walk) an ECPB train to confirm it was intact following certain types of penalty or emergency brake applications.
Overall, the structure of the operating instruction now included separate instructions for:
penalty brake applications – ECPB trains (empty or loaded)
penalty brake application – pneumatic trains
loaded train within loss of brake pipe below 425 kPa at specific locations
empty train travelling 50 km/h or more
all other cases of pneumatic train penalty brake applications.
In addition, a flow chart was included at the back of the document to help drivers select the appropriate procedure.
OI 17-09 also included several changes to address misunderstandings or misinterpretations that had arisen between operations personnel on the meaning of some steps. The requirement to secure all portions of the train with the independent brake and manually apply handbrakes following an emergency brake application with EOT brake pipe displaying ‘off’ remained unchanged.
The publication of OI 17-09 featured in the safe start rail briefing on 1 March 2017. The advice to drivers contained in the briefing stated:
Please familiarise yourselves with OI 17-09 Brake Pipe Emergencies and Penalties, as there has been some changes.
No information was available of the details provided in the associated briefing undertaken by the supervisor and co-ordinator team as part of the safe start briefing. However, the requirement of the rule book module 1 still placed responsibility on persons to whom the rules and procedures applied (drivers) to ensure they were familiar with the instruction upon starting duty.
Operating instruction 17-11
Following an event on 27 March 2017, BHP identified a condition where the ore car brakes would release and, if not otherwise secured against movement, could introduce the potential for a train to roll away (see CCD shutdown event in March 2017).
On 5 April 2017, OI 17-11 became effective, rescinding OI 17-09. The new instruction changed the rule for securing an ECPB train with an emergency brake application (TBC = 120%) with the EOTM displaying ‘off’ (section P6-4.1.1) to include placing the automatic brake handle in the pneumatic emergency position (Figure 14). The instruction, presented in a red font, required drivers to:
Secure all portions of the train by placing the Automatic Brake Handle in the full Emergency Position (dump BP [brake pipe] air), fully applying independent brakes and manually applying handbrakes as confirmed by Train Control.
BHP were unable to retrieve records of the consideration or implementation of procedures for issuing and receiving operating instructions, or a notification of change to ONRSR.
The publication of OI 17-11 may have featured in a safe start rail briefing following the publication of the instruction. However, BHP was unable to retrieve records of the briefing detailing the information provided to drivers following the release of the operating instruction. BHP provided the ATSB a Rail Operations Personnel Signing Sheet that showed that the driver of M02712 had signed for the receipt of OI 17-11 on 4 April 2017.
Figure 14: Extract from BHP OI 17-11
Extract illustrating formats used to highlight key sections of instruction. Highlighted text in the original. This extract does not include all of the content of P6-4.1.1.
Source: BHP
Operating instruction 18-72
On 3 November 2018, OI 18-72 (see) became effective, rescinding OI 17-11. A full copy of OI 18-72 is provided in Appendix A.
The rule applicable to an ECPB emergency brake application (TBC = 120%) with the EOTM displaying ‘off’ (section P6-4.1.1) was unchanged. The step for securing the train was still presented in a red font and it still included the requirement to place the automatic brake handle in the pneumatic emergency position to dump the brake pipe pressure.
The primary change in content of the new operating instruction related to the communication of information to train control. In particular, the rule applicable to an ECPB emergency brake application (TBC = 120%) with the EOTM displaying ‘0’ or above (section P6-4.1.2) was modified to include a new requirement for the driver to ‘Call EMERGENCY on duplicated lines or where deemed necessary’, before advising train control of other relevant details. This change was highlighted in yellow. There was no requirement for the driver to place the automatic brake handle in the pneumatic emergency position for this fault condition, unless the ECPB fault was unable to be rectified and the driver intended to condition the train for conventional pneumatic brake operation.
The notification of OI 18-72 to operational personnel was via the safe start rail briefings. The safe start rail briefing dated 4 November 2018 referenced OI 18-72, and it stated:
28/10/18 - Can all drivers please re-familiarise themselves with ON 18-72 Brake Pipe Emergencies and Penalties. Please ensure you are providing the correct information to Train Control 120% Emergency, 100% Penalty. This is essential to ensure Train Control employ the right level of protection to the train – 120% - no trains can cross or pass the location, whereas with a Penalty - trains can continue past.[29]
On the morning of 4 November 2018, prior to the completion of their night shift, the driver of M02712 became aware of the recent publication of OI 18-72 through a briefing conducted by a supervisor at the Yandi depot office. The driver recalled the supervisor approached them and a small group of other drivers in the office at about 0800 that morning. The supervisor had a printed copy of the operating instruction and took the opportunity to discuss the content and various ‘pick points’ with the drivers present. The supervisor encouraged drivers to get an individual copy of the instruction to read.
The driver of M02712 recollected discussion on the requirement for drivers to contact train control and call an ‘EMERGENCY’ in the case of a brake penalty (section P6-4.1.2). The emphasis placed on the emergency call resonated with the driver, as previously (OI 17-11) there had not been a requirement to call an emergency in response to that ECPB condition; they just needed to advise train control of the penalty and provide pertinent details. The driver recalled no discussion or reminders regarding the automatic brake handle position.
The driver of M02712 did not receive or download a copy of OI 18-72 before starting their next night shift during the evening of 4 November 2018.
Formatting of rules and instructions
With regard to the presentation of the procedural steps and other information, the rule book included a small number of notes throughout. Some were surrounded by a box with a thin red line, some surrounded by a box with a thick red line, and some surrounded by a box with a thick red line together with a yellow caution symbol. Caution and warning symbols were commonly used in other BHP manuals and instructions (see also Procedures for handbrake application and release).
With regard to the operating instructions relating to brake pipe emergencies:
OI 16-16 included some notes in boxes with a thin red line. One of the notes stated that it was ‘essential that the train is secured against movement before repairs are carried out’, and none of the other notes specifically related to the circumstances of the M02712 runaway. A small number of the procedural steps throughout were presented in bold text.
OI 17-09 included some notes in boxes with a thick red line. These boxes contained some additional content relative to the notes in OI 16-16, and they were now all highlighted in yellow. A small number of the procedural steps, and parts of some other procedural steps, were presented in bold text. This included the requirement for a driver to ‘call emergency’ in the case of an ECPB train following an emergency application with EOT brake pipe displaying ‘off’. Red text was now used throughout to refer to TBC values of 120% (consistent with how that information was displayed on the FIRE screen).
OI 17-11 was in a similar format to OI 17-09, with the only change associated with the procedural step in P6-4.1.1 relating to securing all portions of the train. This step was presented in red font, including the parts that were new (automatic brake handle position) and the parts that were unchanged relative to the previous instruction (locomotive brake and handbrakes). The context of the red boxes was unchanged, and they were no longer highlighted in yellow. Underlining was now used for one phrase in one of the boxes.
OI 18-72 was in a similar format to OI 18-72, although section headings were now in dark blue text (as opposed to black) and had slightly different wording. Most of the red boxes were again highlighted in yellow, although the content had not effectively changed (other than changing some words to capitals in some cases). As previously noted, the primary change of content was in P6-4.1.2, where the first step was changed from advising train control to calling emergency. This step was highlighted in yellow. In addition, the equivalent steps in P6-4.1.1 and in a later section relating to pneumatic trains, which were unchanged, were also now highlighted in yellow. A procedural step in P6-4.1.2 relating to positioning of the automatic brake handle to the pneumatic emergency position (if the ECPB was unable to be rectified) was now presented in red text. This was not a new requirement, but the wording had been changed to make it consistent with the wording in P6-4.1.1. Some other minor changes in the text of procedural steps throughout were not highlighted in any manner.
No caution or warning symbols were used in any of the operating instructions. The nature of the changes in each instruction were not summarised within the instruction (like a version history), and no standard symbology, such as a line down the side of an instruction, was used to indicated changes to a previous version.
Operator manuals and instructions
In addition to the rule book and operating instructions, BHP published a suite of operator manuals and instruction sheets for wired distributed power (WDP) operation and the WABTEC/NYAB type ECPB systems. The documentation contained information on the setup/shutdown parameters and trouble-shooting guides for drivers to action in response to various fault conditions.
The ECPB – WDP – Leader Operator’s Manual provided detailed information and flowcharts describing the required actions by a driver in response to an uncommanded ECPB emergency brake application with EOT brake pipe displaying ‘off’. The manual instructed a driver responding to this condition to ‘secure all portions of the train with the independent brake and manually apply handbrakes as confirmed by train control’.
Although the ECPB – WDP – Leader Operator’s Manual was provided by BHP as the latest version, the general information, flowcharts and clauses relevant to brake pipe emergencies and penalties extracted from the rule book did not consistently reflect the amendments detailed in later versions of the operating instruction (that is, OI 17-11 and OI 18-72).
Driver competency assessment related to rules and procedures
According to BHP records, the driver of M02712 underwent training and assessment for ECPB and WDP systems in January 2015, and also subsequently underwent an on-the-job assessment in June 2016.
BHP managed train driver competencies for rail operations through training and assessment processes aligned to the requirements of the Australian Qualification Training Framework unit of competency TLI42615 Certificate IV Train Driving qualification. Any person required to access the rail network received an induction and, where appropriate, additional training in relation to the rail operations safeworking rules, procedures and work instructions. Qualified train drivers underwent scheduled driver reaccreditations (typically at 3-year intervals) to assess their ongoing level of competency related to safeworking and locomotive systems.
The driver of M02712 last underwent the reaccreditation assessment on 25 August 2018 and was assessed as competent. The written (theory) assessment at that time consisted of a number of sections addressing various functions of rail operations, including aspects associated with ECPB. Each section had a series of multiple choice and short answer questions that a driver undergoing accreditation was to complete.
The written assessment included questions associated with the position of controls to set up rollaway protection in the ATP system and the positioning of the automatic brake handle in response to an ECPB-related emergency condition. In relation to these questions:
The ATP rollaway question tested the driver’s knowledge of how they set up the rollaway protection on a locomotive. The driver’s response was to turn the generator field off and centre the reverser. A following question tested the driver’s understanding of when the ATP rollaway penalty would occur, and the driver answered that it would occur if the locomotive unintentionally moved more than 0.5 m. Both of the driver’s responses were assessed as correct.
In relation to the ECPB system, under the section related to rail operations, a question asked the driver ‘In what position does the Automatic Brake handle need to be placed in when performing a walking inspection of an ECPB train with the EOT BP [brake pipe] showing “OFF”?’ The driver’s response stated ‘emergency’, which was assessed as correct.
The next question asked the driver ‘When an ECPB train is required to be secured after an Emergency brake application and the EOT BP is showing 0 or above, the train shall be secured by…?’ The driver’s response stated ‘interlock’ and ‘independent’, which was assessed as correct.
Later in the assessment, in a section related to ECPB and WDP, a series of questions were included that related to a case study involving ECPB where a driver’s loaded train had come to a stand ‘on a grade due to a loss of EOT Beacon Emergency’ with the EOTM displaying ‘off’ on the FIRE screen. In contrast to the emergency involving M02712, the context of the case study was that the driver could not restore ECPB and decided to convert the train to conventional pneumatic braking mode to continue the journey. One question asked what position the automatic brake handle should be placed in before walking the train, and the driver stated ‘emergency’. A follow-up question asked what event this action was to prevent, and the driver stated a ‘runaway’. Both of the driver’s responses were assessed as correct.
Driver experience of brake pipe emergencies and penalties
The driver of M02712 recalled experiencing penalty brake applications on other train services that brought the train to a stand. On those occasions, the fault conditions were different to that involving M02712, and the driver was able to readily recover the penalty application and continue the journey.
The driver could not recall having previously experienced a brake penalty with a complete loss of trainline communications or having to implement the procedures related to a 120% TBC and EOT ‘off’ condition. BHP kept records of reported ECPB-related failures and the associated BHP investigations. For the period October 2014 to 4 November 2018, there was no record of the driver of M02712 previously experiencing an ECPB-related occurrence.[30]
During interview, the driver stated:
Their understanding was that ECPB set to 120% TBC with the interlock on would hold the train in position.
In addition to the interlock, the driver’s understanding was that, by placing the reverser to the centre position and turning the generator field off, they had set up the locomotive rollaway protection provided by the on-board ATP system.
BHP had provided drivers with numerous new operating instructions related to brake pipe emergencies and penalties in recent years, many with only minor or subtle wording changes.
With the change to the operating instructions to require the automatic brake handle to be set in the pneumatic emergency position, there was no explanation provided regarding the reason for the change or why this action was important. They recalled becoming aware of the reason for, and importance of, this action after the occurrence.
There was also no information provided to drivers regarding the ECPB system’s 60-minute shutdown feature.
In relation to the events involving M02712 on 5 November 2018, the driver also stated:
When conducting the tasks associated with the operating instruction, the driver relied on their memory of the required tasks. They did not have a copy of the operating instruction, and there was no process in place for another person to verify that the required actions were conducted.
Due to the nature of the brake pipe emergency and the location, the driver knew before talking to train control that they would need to secure the train and apply handbrakes to the whole train. They also knew that this would take a significant amount of time to achieve.
The driver wanted to expedite the rectification of the loss of trainline communications problem as soon as possible. After reboarding the train to give the train controller the train’s exact position and waiting for their advice regarding handbrakes, the driver was putting on their gloves and getting ready to exit the train to start applying the handbrakes. Accordingly, they did not spend much time checking that they had completed all their required tasks. However, given their knowledge of the ECPB system at the time, the driver believed that taking more time would probably not have resulted in changing their actions to include moving the automatic brake handle to the pneumatic emergency position.
Related occurrences involving brake pipe emergencies and penalties
Following the runaway occurrence involving M02712, BHP audited records of ECPB emergency and penalty events reported between June 2017 and November 2018, finding events where a 120% TBC occurred with a loss of EOT beacon and EOTM displaying ‘off’. The audit selected 63 events (including the occurrence involving M02712) for further analysis.
From those 63 events, BHP focused on occasions when the reported duration to recover the fault exceeded 60 minutes. Of those events, BHP reviewed the degree of compliance with the operating instruction requirement to place the automatic brake handle in the pneumatic emergency position.
Of the 14 events selected, BHP found that for 5 events the driver had applied the emergency brake as per the operating instruction (by placing the automatic brake handle to the pneumatic emergency position). For the remaining 9 events, the driver had not placed the automatic brake handle in the emergency position. Six of these 9 events (including that involving M02712) occurred at a location where the potential for a runaway was present and the risk of a derailment of the train increased.
The BHP database entries against the 6 ECPB emergency and penalty events included details of the follow-up investigation/enquiries that occurred following each event. Other than for the M02712 occurrence, each entry documented a check of the locomotive event recorder logs, finding that a loss of EOT occurred and noting no further action (NFA). The database field for documenting the findings from the investigation/enquiries had no information entered. There was no indication that the extent to which the driver complied with the relevant procedure was examined during the investigations.
BHP’s internal investigation report into the M02712 runway occurrence concluded that there was ‘a perception [in the organisation] that the ECPB interlock will hold the train secure’. The report also noted that the importance and criticality of placing the automatic brake handle into the pneumatic emergency position when responding to a 120% TBC with EOT displaying ‘off’ was not understood as a safety-critical task by all relevant employees.
At the time of the occurrence, the event recorders on BHP’s locomotives recorded parameters related to brake operation and the position of the automatic brake handle. However, information from the recorded data was not being extracted and examined in a systemic way to monitor compliance with the requirements of the braking procedures on OI 17-11 or OI 18-72.
Rules and procedures for securing trains before conducting work
Three-step protection process
In addition to the procedures for responding to brake pipe emergencies and penalties, BHP also had other procedures for securing a train prior to commencing work on the train.
In particular, the Rail Rule Book module 1 contained rules and procedures for a three-step protection process. This process was designed to minimise the risk of injury to workers conducting work on a train or rail vehicle by conditioning the train/vehicle to prevent any unintended movement.
The general rule stipulated:
Before any worker/s enter the profile of stationary rolling stock to perform a task (e.g. handbrake application, inspection, adjustment etc.) except in controlled workshop conditions, the worker/s shall ensure that Three Step Protection has been applied.
The general procedure provided additional information detailing the respective responsibilities of drivers in conditioning the locomotive, and a worker or work team when entering the profile of the rolling stock. The procedure stipulated that the driver was required to:
- apply independent brakes and where required apply the train [automatic] brakes to ensure rolling stock remains stationary;
- place the reverser lever in the neutral position;
- open the generator field switch;
- where the gradient of the track and/or weight of the train may allow the vehicles to move, the train [automatic] brake shall be applied.
The first 3 dot points represented the 3 steps, with the last dot point being an additional action required in some cases.[31]As previously stated, centring the reverser and opening the generator field switch (or turning it off) applied the ATP system’s rollaway protection.
A single worker intending to enter the profile was required to:
- contact driver and verbally request Three Step Protection to be applied; and
- do not enter the profile of the rolling stock until receiving verbal confirmation from the driver that Three Step Protection has been applied.
Where 2 or more workers were required to access the rolling stock profile, the worker responsible for the workgroup needed to perform the above tasks before allowing other members of the group to enter the rolling stock profile.
Application of three-step protection for M02712
After M02712 stopped at Garden, the train controller instructed the driver to apply 101% handbrakes to secure the train against the grade. They also advised the driver that the Redmont maintenance gang would attend to aid the driver. The application of handbrakes by the driver and the group of workers from the Redmont gang required them to enter the rolling stock profile, and therefore apply three-step protection.
After the Redmont gang arrived at the 210 km mark, communications between train control and the gang centred on formulating a plan for the gang to commence at the rear of the train to confirm its integrity, and then commence applying handbrakes from the rear. Subsequent communications between the train controller and driver indicated that the driver was aware the gang had arrived and that the driver agreed with the plan. The train controller instructed the gang and driver to liaise directly with each other.
There was no radio contact made between the driver of M02712 and the gang to confirm the application of the protection. The driver later stated that they had applied the protection anyway, by applying the independent brakes, placing the reverser in the neutral position, and opening the generator field switch. In addition, they also had an automatic brake application due to the ECPB 120% TBC.
The event logger extract for locomotives 4420 and 4472 (Figure 8) recorded the driver’s operation of the independent brake handle at 0340 and positioning the reverser lever into neutral on locomotive 4420 at 0351, prior to the driver starting to apply handbrakes.
The three-step protection process also included the requirement to apply the train (automatic) brake where the gradient of the track and/or weight of the train may allow the vehicles to move. The application of the automatic brake would typically involve the driver moving the handle to a location within the service brake zone. With the train configured in ECPB mode, the operation of the automatic brake handle would result in a 10 to 100% TBC being applied. The level of brake application would likely be determined by the driver dependent on the track grade at that location. For any service brake application with the train configured for ECPB operation, the brake pipe would remain charged.
In the context of M02712, the driver had positioned the automatic brake handle to provide a 39% TBC braking effort along the train. The subsequent disconnection of the trainline cable and resultant emergency 120% TBC response essentially applied full (100%) ECPB effort along the train. With the 120% TBC and emergency interlock active, the driver moving the handle further within the service zone would have had no effect on the braking effort applied to M02712 at the time the driver started applying the handbrakes.
Audits of three-step protection
BHP audited its workers’ implementation of the three-step protection process on rolling stock at various sidings and yards. Auditors recorded findings against requirements detailed in the related audit form. An audit tested workers’ compliance with wearing appropriate personal protective equipment and their fitness for work, and posed a series of questions to workers directly involved in a task to determine their understanding of the process. The audit could also involve a review of downloaded train control radio voice recordings and locomotive event logger data to show the correct application of processes.
In conjunction with the review process for BHP material risks, such as the RME interaction with people, auditors (usually front-line supervisors) also undertook critical control observations to assess workers’ understanding of the hazard present and the effectiveness of the associated critical control, such as the three-step protection process.
BHP provided records of various audits and critical control observations undertaken in 2017 and 2018. Each audit and observation recorded that the associated workers had complied with the requirements of the three-step protection process.
Procedures for handbrake application and release
The BHP work instruction 0105931 (Handbrake application and release mainline recovery) provided information for the safe application or release of handbrakes on trains working on the main line. The instruction cross-referenced critical controls contained in related rules in the rule book, together with information published in operating notices and operating instructions, as well as other work instructions.
The work instruction included the mandatory requirement for the person responsible for the worksite to contact the driver to obtain three-step protection before approaching rolling stock to commence work.
Version 2.0 of the instruction, issued in August 2018, also included a caution for the driver of an ECPB train with the EOTM displaying ‘off’ that, before implementing the three-step process, the automatic brake handle was to be set to the emergency position. The warning, highlighted in yellow and marked by a caution sign symbol, stated:
If the train is conditioned for ECPB operations and the EOTM is ‘off’ or has to be returned to pneumatic brake operations the train driver shall place the automatic brake handle to ‘emergency’ prior to the application of the three step process.
The caution reflected the requirement published in OI 17-11 when responding to brake pipe emergencies and penalties.
Prior to the runaway of M02712 on 5 November 2018, the Redmont gang attended at Garden to aid its driver in applying handbrakes to secure the train. The Redmont gang communicated with train control on arrival but not directly communicate with the drivers of either train M02712 or M02727 to confirm that the driver of the train they were meant to be attending (that is, M02712) had applied three-step protection.
Responders from the Redmont gang consequently entered the rolling stock profile and commenced applying handbrakes to M02727 without confirming the correct application of the three-step protection, potentially exposing themselves to an increased risk of injury.
The M02712 loss of trainline communications occurred at 0338 and the Redmont gang advised train control at 0422 that they had arrived at the (incorrect) train. BHP personnel advised that, even if the Redmont gang had attended the correct train and started applying handbrakes soon after, they would not have had time to apply sufficient handbrakes to secure the train within the ECPB system’s 60-minute shut-down period for the ore cars rear of the point of break in the trainline (that is, prior to 0438).
Emergency management of a runaway train or rail vehicle
Procedures for notifying a runaway
The rule book module 6, section P6-7.0, detailed the actions required from workers and the train controller in response to a runaway of a train or rail vehicle. The worker noticing the runaway was required to:
- transmit an emergency radio message;
- inform the following staff,
- train controller
- any train in the area
- workers working in the area
- take any action necessary to protect trains, other workers and members of the public provided it can be done without further increasing risk to self and / or others.
The procedure for a runaway addressed an event associated with an uncontrolled or uncommanded movement. Although pertinent to a critical control for emergency response following such an event, the procedure was not referenced within the mitigating controls linked to the material risk of rail mounted equipment interaction or other material risk identified within the BHP risk management system.
When train M02712 commenced to roll away at 0440, the train controller became aware of the movement through observing track occupancy indications displayed on the train control monitor, as well as radio communication with the driver of the empty east-bound train M02727 at 0444.
As noted in The occurrence, the driver of M02712 notified the controller of the runaway at 0446. The driver had lost their footing when the train began to move, slipping on the ballast formation and knocking their radio off channel. After resetting the radio, the driver contacted the train controller, declaring an emergency and notifying of the runaway.
Train control response to emergencies
Following receipt of an emergency radio message from a worker alerting a train controller of a runaway involving a train or other rail vehicle, the rule book module 6 procedure required the train controller to take any necessary action to protect trains, other workers and members of the public.
The train control incident and emergency response procedure[32] further outlined the role and responsibility of the train controller in undertaking the initial response and management of an incident or emergency that occurred on or in the proximity of the BHP rail network.
The procedure addressed responses to an incident or emergency involving injury to a person on the BHP rail network, at an interface point with a third party such as a level crossing, or between BHP and another rail transport operator’s network. For incidents or emergencies involving BHP rail operations, the procedure required the train controller to implement a multi-step process involving the:
exclusion of rail traffic
gathering of relevant information
recording of details on the Train Control Graph
implementing processes to protect the sites and assist field staff
completion of the rail safe working notification form.
For defined types of events, the procedure included an escalation protocol matrix for notifying supervisors or superintendents as required. An escalation event included:
any event that was classified as Category A reportable incident to ONRSR
any accidents, incidents or near misses that occurred at a rail interface with a third party, or the area within 5 m either side of the applicable railway tracks and associated track structures, irrespective of whether injury to a person or damage to property or equipment resulted.
An event such as the runaway of train M02712 therefore required immediate escalation to a superintendent level.
The emergency response procedure also linked to the two-part emergency management plan,[33] which described the arrangements BHP iron ore operations put in place to prepare, respond and recover from an emergency event. These plans defined the arrangements for various field response and corporate support teams based on the location and type of event.
In responding to the notification of the runaway of train M02712, the train controller excluded trains approaching the location of M02712 on the adjacent track, gathered relevant details of the event and arranged for resources to assist the driver. The train controller also placed trackside signals immediately in front of M02712’s movement at stop in an attempt to trigger ATP and stop the train.
The train controller based the initial response on the principle that the ATP system would stop the movement of M02712. When this did not occur, train control formed an understanding (in conjunction with advice from other drivers) that M02712 would slow and come to a stand on the rising grades approaching Woodstock (approximately 154 km from Port Hedland). The controller continued to manage the safety of the affected train on the adjacent track and the welfare of the driver of M02712.
About 30 minutes later, when it became evident to train control that M02712 was not stopping, the train controller contacted the drivers of the other trains operating ahead of M02712, instructing them to also stop, secure their trains and move to a position of safety.
Interface coordination plans
The Newman to Port Hedland railway was located predominately within pastoral leases, local government and BHP property. A number of interfaces existed between BHP and adjoining rail infrastructure managers (RIMs), pastoralists/local government and road managers of public and private roads crossing the railway. The interface agreements defined the respective roles and responsibilities of each party and the provisions put in place to manage risks to safety identified at the various infrastructure or operational interfaces.
The section of the railway between the Garden (211 km) and Turner South (119 km) contained the following infrastructure interface points:[34]
rail/rail interfaces
Fortescue Metals Group 148.8 km, Cloudbreak rail overpass
Fortescue Metals Group 186.75 km, Solomon rail overpass.
rail/road interfaces
2 km, passive level crossing
4 km, active level crossing – Roy Hill to Munjina Road, public access
The interface agreement for the overpasses at Cloudbreak and Solomon involved 2 other RIMs. The agreement detailed the parties’ responsibilities and the agreed arrangements for notification and management of risk associated with inspection or maintenance activity and in response to an incident or emergency.
The risk assessments identified the potential for damage to the rail overpass abutments (and potentially the integrity of the track above) arising from various factors, including derailment of a BHP train due to an overspeed. The risk assessments also identified various BHP controls to mitigate the risk, with the residual risk assessed as high.
The interface agreement stipulated that parties immediately report to each other any accident, incident or a near miss that occurred at the interface, irrespective of whether injury to a person or damage to property or equipment resulted. Similarly, following the receipt of an emergency notification from operational staff, the respective train control personnel were to advise the other parties train control personnel immediately in accordance with an activity and notifications table included in the agreement. The notification table identified a life threatening or operational disruption event, such as train derailment, track obstruction, level crossing or infrastructure damage. The description did not define the notification required in response to an emergency involving an uncontrolled train movement (or runaway).
With regard to the occurrence involving M02712, BHP staff at the integrated remote operations centre in Perth communicated with affected BHP staff to manage the event. Although relevant contact details were included in the train control and emergency response procedure, BHP staff did not contact representatives of the RIMs that had interfaces with BHP to warn of the potential risk at the interfaces from the runaway of M02712.
Fatigue management
Regulatory requirements and guidance
Given that the train stopped at 0340 and the driver was conducting a series of 7 night shifts, the ATSB examined BHP’s processes for managing train driver fatigue.
BHP developed its fatigue management procedures to be consistent with the Code of practice: Working hours, issued by the Western Australia Commission of Occupational Safety and Health and its Mining Industry Advisory Committee in 2006.[35]The code stated that it addressed:
… issues that might potentially arise in some working hours arrangements, for example extended hours, shiftwork and on call work. It brings together a range of recognised workplace hazard factors that must already be addressed, as far as practicable, where there are occupational safety and health risks…
As individual workplaces and industries have different working hours arrangements, this code of practice provides high level general guidance and recommendations on risk management. It is suggested that the risk management approach is tailored specific to the unique demands of each workplace and/or industry.
The code provided guidance for employers to identify fatigue-related hazards, assess their risk and implement risk control measures. It outlined a significant number of potential hazard factors to consider. The guidance indicated that factors associated with higher risk for shiftwork, such as fly-in fly-out (FIFO) working arrangements, included 12-hour shifts, night shifts, backwards rotation, slower rotation (such as weekly), 7 sequential 12-hour night shifts and extended travel prior to starting a FIFO shift pattern. There were no specific requirements or minimum standards for roster arrangements.
From November 2015, the Office of the National Rail Safety Regulator (ONRSR) administered the Rail Safety National Law (WA) Act (2015). The Act stated that a duty of a rail transport operator included ensuring that:
…rail safety workers who perform rail safety work in relation to the operator’s railway operations do not carry out rail safety work while impaired by fatigue or if they may become so impaired
Operators were also required to have a safety management system that included a fatigue risk management program. The Rail Safety National Law (WA) regulation 29 stated a range of things an operator had to take into account when preparing a fatigue risk management program. These included (but were not limited to) the scheduling of work and non-work periods, the time when work was undertaken, the length and frequency of rest breaks, circadian effects, chronic sleep loss effects, the types of rail safety work being performed, the suitability of rest environments, the physical environment, and relevant developments in research related to fatigue. There were no specific requirements or guidance regarding the design of shifts and rosters.
ONRSR advised that, for the period from November 2015 until November 2018, it had no records of any notifications of change, variation to accreditation or any regulatory oversight activity conducted in relation to train driver rostering practices for the operator.
Overview of operator’s procedures
The BHP Western Australia Iron Ore (WAIO) fatigue management procedure (SPR-IHS-SAFOH-004) outlined BHP’s approach to controlling the risks associated with fatigue. It applied to all BHP iron ore employees and contractors, including train drivers.
The scope section of the procedure stated:
Fatigue presents a material risk in Iron Ore. This procedure is founded on the WA Code of Practice and Working Hours, which itself is based on extensive studies in the field of fatigue risk management and control. As a result there is a significant base of science underpinning BHPBIO fatigue risk management requirements.
Causes of fatigue include but are not limited to:
- Roster design and working hours
- Work tasks and environment
- Amount and Quality of sleep
- Sleeping environment
- Sleep disorders and other health issues….
Fatigue should be managed at the following levels and in the following order:
- Self-Management
- Peer Management
- Supervisor Management
It must be recognised that the individual has the greatest amount of control over their own fatigue and as such the primary accountability for managing personal fatigue risk resides with the employee.
In addition to the fatigue management procedure, additional requirements and guidelines were outlined in related documents. In particular, the WAIO approved rosters procedure provided requirements for planned rosters. It also noted that fatigue could occur due to:
- Too little or poor quality sleep;
- Working during normal ‘sleep’ times;
- Carrying out mentally or physically demanding activities; or
- Other health factors.
Rostering rules/principles
The WAIO fatigue management procedure stated:
The following key fatigue risk factors are relevant to day to day operations and represent the threshold conditions, beyond which additional controls may be required to manage fatigue risks.
3.2.1 Maximum working time per 24 hours will not exceed 14 hours, inclusive of travel time.
3.2.2 Maximum of 14 consecutive dayshifts
3.2.3 Minimum time between shifts to not be less than 10 hours.
3.2.4 When rotating shifts, they are to rotate from day to night.
3.2.5 When rotating from day shift to night shift, the minimum time between shifts is not less than 23 hours. For avoidance of doubt, the transport time to and from the point where work commences can be within the 23-hour period
3.2.6 Maximum of 7 consecutive nightshifts for FIFO, 4 consecutive nightshifts for residential.
3.2.7 Where a roster starts on nightshift, flight arrangements will ensure that individuals have the opportunity for 4 hours sleep at their site accommodation before the start of their first night shift.
The WAIO approved rosters procedure also specified a set of ‘fatigue rules’, against which planned rosters were required to be assessed prior to approval. For FIFO operations, these rules included:
maximum normal shift length – 12 hours (excluding shift handover)
maximum shift handover – 30 minutes
maximum number of consecutive shifts – 14
maximum consecutive night shifts – 7
rotation of roster – forwards (day shift followed by night shift)
minimum rest period between shifts – 10 hours
break between day and night shift (or night and day shift) – at least 23 hours.
A night shift for rail operations (with continuous rolling roster patterns) was defined as working more than 3 hours between the hours of 2300 and 0600 (that is, any 12-hour shift starting between 1500 to 0200).
Specific management approval was required for rosters that exceeded any of the ‘threshold conditions’ or ‘rostering rules’. With regard to FIFO train drivers, an approved exemption had been in place since August 2013 that allowed for roster patterns or ‘swings’ to start with night shifts followed by day shifts.
The operator’s FIFO train drivers typically worked swings that consisted of 7 12-hour shifts (each starting at the same time), a 24-hour recovery break, 7 12-hour shifts (each starting at the same time) and then 12 days off duty. The start time of the first shift would vary for each swing. For example, for the driver of M02712 during 2018, the first shift of their swings started at the following times: 0300, 0600, 0200, 2200, 0500, 0200, 0600 (training), 2100, 0400, 0000 and 2200. The swings commencing at 2100 to 0200 were classified as starting with night shifts, whereas the swings commencing at 0300 to 0600 were classified as starting with day shifts.
Fatigue monitoring
BHP provided operational staff and their managers with a ‘fatigue assessment tool’. The tool asked a worker to provide answers to a series of 8 questions related to sleep and alertness, including: hours sleep in the last 24 hours, hours sleep in the last 48 hours, hours awake at the end of the shift, perceived level of alertness, use of alcohol or medications that could cause drowsiness, and health or personal problems that could influence concentration or sleep.[36]
The answer to each question could be assessed as low (0 points), medium (1 point) or high (2 points) risk, and the overall score across the 8 questions could be then assessed as low risk (0–2 points), medium risk (3–7 points) or high risk (8 or more points). For example, low-risk scores for sleep included at least 7 hours sleep in the last 24 hours and at least 14 hours sleep in the last 48 hours, and high-risk scores included less than 5 hours sleep in the last 24 hours and less than 12 hours sleep in the last 48 hours.
In the case of an overall medium-risk score, the recommended actions for supervisors included rotating tasks, encouraging the use of alertness strategies, providing opportunity for short breaks, having personnel work together, or removing the person from safety-sensitive work. In the case of an overall high-risk score, recommended actions included immediately preventing the person from working and determining if the individual could be placed on alternative duties.
The fatigue assessment tool was required to be used when a worker reported they were ‘fatigued’, a supervisor or peers observed signs of fatigue, or other situations ‘where there may be fatigue risk’. FIFO train drivers were also required to complete the tool prior to their second shift and their ninth shift on each roll-over swing.
An independent review of fatigue management at BHP WAIO rail operations completed in February 2020 (see also Sleep studies, surveys and other assessments) noted that the way BHP’s tool derived an overall risk score could underestimate the level of risk because a high score on multiple questions could still result in only a medium-risk score overall.
In addition, the review noted that a number of people interviewed had stated workers were reluctant to self report fatigue, and few workers were willing to complete the fatigue assessment tool accurately or provide responses that would raise a supervisor’s concern about their fitness for work. Concerns were also raised about how workers who self reported fatigue were managed.
BHP advised that, during 2018, 12,860 fatigue assessments were completed. The vast majority (97.6%) resulted in low-risk scores, with 297 (2.3%) resulting in a medium-risk score and 6 (0.04%) resulting in a high-risk score.
The driver of M02712 completed a fatigue assessment tool prior to commencing their second night shift on 31 October 2018. Only summarised information from completed forms was stored by BHP, and the result for 31 October indicated a low risk. BHP reported that the driver’s other assessments for 2018 also produced low-risk scores.
The driver stated that they had never self reported being fatigued while working at BHP. They also noted that when a driver did not pass the fatigue assessment tool they were not allowed to drive a train; instead they were generally kept around the office to do administrative tasks (including driving other drivers to their trains), which was perceived negatively by the drivers.
Fatigue management training
BHP’s fatigue management procedure stated that ‘Appropriate education and training shall be provided to assist in the prevention and management of fatigue.’
The independent review of fatigue management completed in February 2020 stated areas of concern regarding fatigue management training. It noted that individuals who had recently conducted induction training reported that the content on fatigue was basic in nature. The review also noted that, according to a BHP manager, courses on ‘fatigue education’ and ‘night shift and fatigue management’ were available on the BHP’s learning management system: however, a sample of employees and supervisors appeared to be unaware of these courses or whether they were mandatory, and none had undertaken such a course recently.
The driver of M02712 completed fatigue awareness training course in May 2009. They also completed an on-line recurrent training course in February 2017 that covered fatigue awareness aspects (amongst other safety topics). The content of the 2017 training including advice on techniques to minimise fatigue and assist with sleep. It also stated that ‘you have to rest for at least 10 hours between shifts, so you have a chance of sleep’. The training did not provide advice on the minimum amount of sleep required between shifts, but noted that if a worker was worried about their level of fatigue they could use the fatigue assessment tool to determine their level of fatigue.
Accommodation on site
FIFO train drivers were provided with airconditioned accommodation in a unit having its own bathroom. The rooms were described as providing good control over noise, light and temperature.
The driver of M02712 reported the sleeping accommodation was suitable. The driver also noted that the set meal times and meals at the depot were not conducive to a night shift roster, with options available at 1730–1800 being more traditional dinners rather than being the sort of meals people would normally eat for breakfast.
Additional fatigue risk controls and mitigators
As part of its risk management process, BHP had first level supervisors conduct critical control observations (CCOs) related to fitness for work and fatigue management. These involved a front-line supervisor asking a worker 3 questions about fatigue management aspects (including the process for reporting fatigued and the tool available for assessing fatigue). Overall, 4 CCOs were conducted in 2018 on BHP’s train drivers. No problems were noted in the drivers’ understanding of the matters assessed.
BHP also conducted one internal audit of fatigue management aspects within its rail operations in 2018. The audit primarily assessed workers’ awareness of (and compliance with) requirements related to not exceeding 14 hours work a day.
With regard to FIFO travel, as noted in the rostering rules, a worker was required to have a 4-hour sleep opportunity after arriving at camp and prior to commencing their first night shift. The fatigue management procedure also stated that:
It is an individual’s responsibility and duty of care to manage their own fatigue and their commute arrangements to the airport in order to safely work their full first shift…
Reasonable flight arrangements will be scheduled to enable individuals to manage fatigue.
The fatigue management procedure and the fatigue rules did not specify any minimum requirements regarding rest breaks within a rostered work period. FIFO train drivers were permitted to stop for a 30–minute ‘crib’ break (or meal break) between 4 to 7 hours into each shift. A review of the train control diagram for the morning of the accident noted that drivers departing from Yandi typically took a 30-minute rest break at Coonarie or Woodstock after conducting about 6.5 hours duty (ranging from 5.5 to 7.3 hours prior to the break).
The driver of M02712 stated that, during a crib break, they either sat inside the locomotive and opened up the window for fresh air or stood on the nose or side of the locomotive, had a meal, then grabbed a fresh bottle of water when resuming driving duties.
BHP procedures and practices included a range of additional process to help minimise the risk of fatigue. These included monitoring of fatigue effectiveness management by supervisors, employee assistance programs, drug and alcohol testing, and inclusion of sleep-related items on annual medical examinations (in accordance with the National Standard for Health Assessment for Rail Safety Workers).
Use of biomathematical models of fatigue
A biomathematical model of fatigue (BMMF) uses mathematical algorithms to predict the effect of different patterns of work on measures such as subjective fatigue, sleep or the effectiveness of performing work. Each model uses different types of inputs and produces different types of outputs, and each model is based on many assumptions and has limitations.
In particular, the models are based on group-averaged data, and it is widely agreed that the models are not well suited for predicting a specific individual’s level of fatigue. In addition, none of the models consider all of the factors that can influence fatigue. The models are designed to be one element of a system for evaluating and comparing work rosters (see Civil Aviation Safety Authority 2014, Dawson and others 2011, Gander and others 2011, Independent Transport Safety Regulator 2010).
BHP used the BMMF known as ‘FAID’[37] to conduct assessments of some of its rosters. FAID has been widely used in the Australian rail and aviation industries since the early 2000s. It uses hours of work (start time and end time) as its inputs, and it produces a score based on an algorithm that considers the effects of the length of the duty periods, time of day of the duty periods and the amount of work over the previous 7 days (Roach and others 2004). The more recent the duty period, the more effect the duty period has on the resulting score. The higher the FAID score, the higher the potential for fatigue.
FAID documentation stated scores of 40–80 were broadly consistent with a safe system of work. However, the threshold for deciding the acceptability of a roster needed to be set by the operator based on a fatigue hazard assessment, taking into account the fatigue-related hazards specific to the role or task, and determining the acceptable level of fatigue tolerance for that role or task. Without this assessment, the FAID program defaulted to a fatigue tolerance level (FTL) of 80.
The ATSB requested BHP to provide any fatigue modelling assessments of FIFO train driver rosters. It provided documents that summarised various assessments conducted in 2011–2013. Key points stated in these documents included:
BHP set the FAID FTL at 90 due to ‘the use of WAIO fatigue procedures and controls’.
FAID was not a ‘yes/no’ tool and scores greater than the FTL did not prevent work from occurring. Rather, FAID was a risk predictive tool and when high scores were identified then suitable controls needed to be established and monitored.
FAID identified night shifts as a significant risk and the greater the number of consecutive night shifts then the higher the scores. In comparison, afternoon shifts had lower scores (particularly for shifts ending before or about 0000).
The WA Code of Practice noted a key risk with night shift, specifically during 0200–0600, and ‘any use of targeted or strategic risk controls during this period is recommended’.
The documents included assessments of actual or proposed FIFO swings (with 7 12-hour shifts, 24-hour break and 7 12-hour shifts) with various start times. Results included:
For a typical series of 4 swings with different start times, 23% of the duty time exceeded a FAID score of 90.
All of the swings had multiple shifts with peak FAID scores exceeding 100, and many had multiple shifts with peak scores exceeding 120 (usually on night shifts).
Many of the swings had peak FAID scores in the range of 140–149, with the peak score occurring towards the end of the seventh night shift. Shift start times associated with peak FAID scores of 148–149 included 2000, 2130 and 2200.
Most of the swings were evaluated as meeting the current requirements of the fatigue management procedure and the fatigue rules. Some of the proposed swings, with night shifts followed by day shifts, were identified as not meeting BHP’s procedural requirements and therefore needed a risk assessment and approval before implementation (see next section).
A review of the fatigue modelling documents provided by BHP to the ATSB noted that:
When evaluating new proposed shift times, the documents normally stated that the scores were similar to those found for previous FIFO rosters (rather than state their overall level of risk).
Travel times for FIFO workers prior to their first shift were not included in the modelling. Including travel time would increase the scores in the first few shifts by a small amount.
The shift patterns being modelled had no work hours for at least 7 days prior to the first shift. A key feature of FAID is that it only considers the duty periods over the previous 7 days, with the influence of a duty period decaying over the 7 days. If there were no duty periods in the previous 7 days, then there was a lag as the score in the next duty periods accumulated. Accordingly, FAID scores for the first few shifts after 7 or more days of no duty time will underestimate fatigue levels.[38]
The ATSB notes that FAID scores (and the scores from any BMMF) need to be interpreted with caution. The Independent Transport Safety Regulator of New South Wales (2010) stated that, due to various factors associated with the model, ‘a FAID score of less than 80 does not mean that a work schedule is acceptable or that a person is not impaired at a level that could affect safety’. In addition, the US Federal Railroad Administration (2010) concluded that in some situations FAID scores between 70 and 80 can be associated with ‘extreme fatigue’.
Fatigue risk assessments
A risk assessment was conducted in August 2013 by BHP for the exemption to the threshold conditions and rostering rules that allowed FIFO train drivers to work a roll-over swing with 7 12-hour night shifts followed by 7 12-hour day shifts. The list of ‘current controls’ for this arrangement included:
vigilance control and ATP (on board a train)
fatigue management procedure
fatigue assessments (as per the fatigue assessment tool)
fatigue management training for drivers and supervisors
medical assessments
drug and alcohol testing
employee assistance program
airconditioned accommodation
fatigue breaks
FIFO travel commute plans.
These controls reduced the level of raw risk from ‘extreme’ to a residual risk of ‘high’. No additional risk controls were added to reduce the risk, other than to ensure the roster was approved.
Sleep studies, surveys and other assessments
BHP was asked to provide evidence of any sleep studies, surveys, expert reviews or other reviews conducted for the FIFO train driving rosters (prior to 4 November 2018). No evidence of any sleep studies or surveys associated with the FIFO train drivers was provided.[39]
In August 2018, at the request of BHP, an independent organisation completed a review of WAIO train driver rosters. The review used different criteria to assess residential versus FIFO rosters, noting that FIFO workers generally had a more controlled sleeping environment, reduced domestic and family demands, less competition for sleep times from leisure and domestic pursuits, and reduced daily commute times.
The report concluded that, while 7 consecutive night shifts and 14 consecutive shifts was still very common in the Western Australian mining industry, BHP’s roll-over roster patterns for FIFO train drivers had several significant additional risk factors. High-risk aspects included day shift early start times of between 0000–0300, night shift finish times between 0800–1400, the change from nights to days within a roster pattern (compared to the reverse direction), and the short length of the recovery break (24 hours) between night shifts and day shifts.
Early start times were considered problematic because many employees would find it difficult to get 8 hours sleep due to 1600–2000 being a period of high natural alertness. The late night shift finish times were considered problematic due to environmental and bodily conditions making it difficult for employees to fall asleep and stay asleep during the day. The review also stated that other aspects, such as 7 consecutive night shifts, were considered a medium risk. The report noted that, in combination with the high-risk factors, ‘many individuals are likely to build up a significant sleep debt over these 7 night shifts’.
The August 2018 review outlined several recommendations. These included ensuring that BHP provided recent and comprehensive education to drivers and supervisors and to review whether the additional risks posed by working night shifts prior to day shifts was adequately mitigated by the stated control measures.
The same organisation completed a review of fatigue management at BHP WAIO rail operations in February 2020. That review noted that the exemption which permitted swings with night shifts followed by day shifts did not appear to include the involvement of a fatigue subject matter expert. The review stated that the roster pattern contained ‘a very high risk of fatigue’ and that most of the controls specified in the exemption were administrative in nature and were already in place for approved rosters.
The February 2020 review further stated that commencing with night shifts was problematic as the fatigue accumulated over 7 night shifts was carried over into the day shifts. It was also problematic as it could lead to workers travelling in on the day prior to their first shift and not getting any sleep in the afternoon or evening prior to that first shift, resulting in them being awake for an extended period (more than 24 hours) prior to the end of their first shift. In addition, the review acknowledged that drivers preferred the roster patterns with nights followed by days as it meant that they ended up with an additional day at home.
Information about research into the effects of night shifts and roll-over roster patterns on sleep is provided in Appendix B.
Additional fatigue modelling
The ATSB applied 3 different BMMFs used in the rail industry to a roll-over roster pattern with 7 12-hour night shifts starting at 2200 followed by 7 12-hour day shifts starting at 1000, with no work in the previous 7 days. The purpose of the applications was to understand the nature of the results such models would provide if they had been used by an operator to examine such a roster pattern. More specifically:
FAID was applied using the BHP FAID score threshold of 90. It resulted in 22% of scores above 90 and a peak score of 148. All of the scores above 90 occurred on the night shifts, with the amount of time above 90 increasing from 0.8 hours on the third shift, 7.9 hours on the fifth shift, 9.8 hours on the sixth shift and all of the seventh shift. The highest scores each shift occurred between 0300–0800, and particularly between 0500–0700. The scores between 0300 and after 0800 on the sixth shift exceeded 115, and for the same period on the seventh shift they exceeded 120.
FAID Quantum (available since 2016) was applied using the default threshold Karolinska sleepiness scale (KSS) score of 7 (out of 9).[40] It resulted in 24% of KSS scores above 7 and a peak KSS score of 8.2. All the scores over 7 occurred on the night shifts, with the amount of time above 7 increasing from 6.5 hours on the third shift to 7.5 hours on the fifth shift and 7.8 hours on the seventh shift. The scores passed 7 at about 0300 on each of these shifts and reached a peak shortly after 0600. The model also predicted 4.5 hours sleep following each night shift. This is broadly consistent with available research on the average amount of sleep workers obtain with multiple 12-hour shifts starting at 2200 (Roach and others 2003, Paech and others 2014; see Appendix B).
The fatigue avoidance scheduling tool (FAST) was applied using a no-sleep zone of 1600–1900 (recommended for regular night shift workers) and a commute time of 60 minutes, resulting in 5-hour sleep periods following each night shift. This resulted in performance effectiveness scores[41] that gradually decreased over the 7 night shifts, with the lowest scores in each shift occurring at about 0600. Significant portions of the night shifts were below the criterion line (77.5% effectiveness) and notable amounts were in the red zone (below 65% effectiveness). The percentage of time below the criterion line increased from 66% for the first night shift to 79 % for the last 3 night shifts. The average effectiveness score decreased from 77 for the first night shift to about 64% for the last 3 night shifts.
As indicated previously, BMMFs are not well suited for predicting a specific individual’s level of fatigue at a specific point in time. For the purposes of comparison with the other results above, the score from the various models (using default inputs) for 0345 on the sixth night shift were a FAID score of 120, a FAID Quantum KSS score of about 7.5, and a FAST performance effectiveness score of about 60.
In January 2021, a consultancy organisation provided a report to BHP regarding its rail operations rosters. The review used FAST to conduct modelling of BHP’s current FIFO rosters and some proposed alternatives. With regard to the type of swings being used at the time of the 18 November 2018 accident, the report noted that each swing, regardless of the initial sign-on time, would produce an overall average score (across the 14 shifts) that could be considered ‘extreme risk’ or ‘high risk’. Initial sign-on times associated with the worst average scores were from 2100–0300, with the highest scores being from 0000–0100.
On 5 November 2018, a BHP loaded ore train M02712 was approaching an access road level crossing near Garden South. Shortly after, there was an unplanned interruption of trainline communications, which triggered an emergency brake command to the electronically controlled pneumatic braking (ECPB) system.
The train came to a stop on the -1.5% falling track grade approaching Garden South. The train was subsequently not effectively secured on the falling grade and, about 60 minutes after the ECPB’s system-initiated brake command, the train started rolling away without the driver on board. M02712 travelled for about 91 km before Hedland train control purposely derailed the train at Turner South. The derailment destroyed the 2 remote locomotives, 245 ore cars and 2 km of track infrastructure. There was no injury to any person from the runaway or derailment.
A train runaway can cause injury or loss of life, substantial damage to rolling stock and infrastructure, and disrupt rail operations for an extended period. Accordingly, rolling stock operators and rail infrastructure managers need to implement sufficient preventative and mitigating controls to manage this hazard.
In this case, a number of safety factors contributed to train M02712 not being effectively secured. This analysis will initially discuss some limitations in BHP’s initial integration of the ECPB system with related systems and its subsequent risk assessment of the potential for a runaway event to occur. It will then discuss limitations with the design and introduction of the emergency procedure for responding to a loss of trainline communications, before discussing the loss of trainline communications involving M02712, the driver’s response to the emergency, and the inability of the automatic train protection (ATP) system to stop a runaway train in this situation. The analysis will then discuss a number of other factors that increased safety risk identified during the investigation, including the use of three-step protection, emergency response arrangements and fatigue management.
ECPB system integration
In 2011, BHP commenced the process to implement ECPB on its main line operations due to its many advantages over conventional pneumatically-braked trains, and it elected to implement it as an overlay system in order to provide operational flexibility. The implementation involved integrating the ECPB system with other onboard systems, including the pneumatic braking system and the ATP system.
BHP’s introduction of ECPB involved numerous assessments, trials and other activities over an extended period prior to commencing main line operations with ECPB overlay trains in 2015. However, during this period it did not identify and manage 2 significant characteristics that affected how the ECPB system integrated with these other systems. More specifically, when a train was in service, operating in ECPB mode, and there was a break in the trainline resulting in the end of train monitor (EOTM) displaying ‘off’:
The car control devices (CCDs) rear of the point of break in the trainline would shut down and release their brake application 60 minutes after the loss of communications with the head end unit (HEU) locomotive.
In the event the CCDs shut down, and the train was not effectively secured against unintended movement, any subsequent automatic train protection (ATP) penalty requests to the ECPB system would be ineffective in stopping the uncommanded movement. In ECPB mode, the ATP system detecting the on-going movement could not then cause a dump of brake pipe pressure to initiate a brake application to the train via the brake pipe.
Consequently, BHP’s trains configured for ECPB operation were potentially vulnerable to a runaway event should the following combination of events and conditions occur:
The train was on a descending or ascending grade (which existed at various locations within the BHP Pilbara iron ore rail network).
There was a loss of trainline communications, resulting in the EOTM displaying ‘off’.
The loss of trainline communications occurred towards the front of the train, meaning a greater proportion of the CCDs in the train would shut down after 60 minutes.
The train pneumatic brake pipe remained intact and charged.
The driver did not place the automatic brake handle in the pneumatic emergency position (to dump the brake pipe air).
There was insufficient time for the driver and other personnel to apply the required number of handbrakes to secure the train.
The driver (or other rail safety worker) was not on board the train and able to apply emergency braking pneumatically when the train started rolling away.
This scenario was effectively what occurred on 5 November involving M02712, as will be outlined in more detail below in later sections.
The only control preventing this scenario resulting in a runaway was if the driver, in response to the emergency, placed the automatic brake handle in the pneumatic emergency position. However, up until April 2017 (see next section), this action was not a required part of BHP’s procedure for responding to this type of emergency. Even after the procedure was amended to include this action, the system was still vulnerable as it relied on the effectiveness of a single administrative (procedural) control, and a simple omission by a driver of one action in the procedure could still result in a runaway event.
The specific reasons why the 2 significant characteristics were not effectively identified and/or managed during BHP’s implementation of ECPB were not able to be determined. However, as recognised by BHP itself, the rolling stock operator did not have a systems engineering framework in place during the introduction of ECPB and the integration of ECPB with other related systems. Instead, BHP predominately managed the implementation of its electrically controlled pneumatic brake (ECPB) overlay and modification of automatic train protection (ATP) systems at an individual system level. The use of a systems engineering framework provides a structured approach to manage the risk of designing and implementing complex systems, and increases the likelihood that hazards associated with the integration of a new system with other systems will be effectively identified and managed.
Application of risk management processes
Risk assessment of train runaway events
Although BHP did not effectively use a systems engineering framework during the implementation of the ECPB overlay system to manage the potential risks associated with using that system, it did have established risk management and associated management of change procedures in place as part of its safety management system. These processes involved identifying, assessing and managing ‘material’ (or significant) risks in its operations. As part of this process, critical preventative and mitigating controls for each risk event were identified, which then enabled management to focus oversight on these controls to prevent or mitigate the potential consequences from the risk event.
BHP had identified a rail-mounted equipment (RME) interaction incident as a risk event, and the associated risk assessment (presented in a bow-tie format) showed that one of the potential causes of such an incident was an uncontrolled or uncommanded movement of a train or other RME. Although this risk assessment process provided the opportunity to identify a loss of trainline communications and its response to be a cause of an RME interaction incident, this did not occur. Overall, there were several limitations with the overall nature of the resulting risk assessment. More specifically:
The range of events covered by an RME interaction incident was very broad, including events associated with shunting in yards, conducting maintenance on rolling stock, and operation of hi-rail and maintenance vehicles as well as main line train operations. Such a broad scope increased the potential that insufficient focus would be placed on some specific types of events (such as a train runaway event).
The risk assessment (and the risk assessment process) generally focused on health and safety considerations. Although preventing fatalities is paramount, this focus likely biased the identification and assessment of preventative controls toward addressing the risk of a worker fatality during rolling stock maintenance and repair activities, and limited the focus on a range of other operational circumstances that could give rise to the potential for a train runaway event on the main line.
The range of failures and events that could lead to an uncontrolled or uncommanded train movement was not articulated in any meaningful detail. Although ‘brake failure’ and ‘failure to secure vehicle against movement’ were listed, a wide range of scenarios could result in either, many of which would require different sets of critical controls.
A set of critical controls was identified for the uncontrolled or uncommanded movement of a rail vehicle, and these focussed on the competence of the workers involved, the three-step protection process and rolling stock maintenance. However, there was no reference to the procedures associated with responding to a brake pipe emergency or penalty. In addition, none of the controls appeared to specifically focus on preventing a train runaway on the main line.
None of the causes or critical controls focussed on the ECPB system or the integration of this system with other related systems, such as the ATP system.
The ATP system failing or being overridden was listed as one of the potential causes of an RME interaction incident, but there was no reference to potential conditions where the ATP would operate as designed but be ineffective. ATP was also included as a critical preventative control, but not for the prevention or mitigation of an uncontrolled or uncommanded rail movement. Overall, ATP appeared to be treated as an individual system, and potential limitations in its interaction with other systems were not articulated.
Vendors modifying the ATP system for the dumper automated spotting of locomotives (DASL) system project submitted a series of safety-related application conditions (SRACs) to BHP in 2013, 2015 and 2016, which included reference to various conditions that could potentially lead to unintentional train movement. There was no indication that this safety-related information was integrated into the risk assessment.
Ultimately, the risk assessment served an important role in identifying important critical controls and ensuring appropriate management attention was focussed on verifying the integrity of these controls. However, the broad scope and general nature of the risk assessment in this case was not well suited for identifying and managing risk associated with the integration of complex systems, and it was not well suited for managing the risk of an ECPB train runaway.
Management of critical controls for responding to brake pipe emergencies
On 27 March 2017, the driver of another loaded ore train reported an ECPB 120% penalty brake application at Garden with the EOTM displaying ‘off’. The driver responded to the event by generally following the procedure for responding to brake pipe emergencies and penalties that was applicable at that time, contained in operations instruction OI 17-09.
Although a runaway did not occur on that occasion, later investigation by BHP maintenance personnel found that ore car CCDs would release their brake application under certain conditions. Information about the 60-minute shut-down feature of the CCDs, and the need to either secure the train or cut out the ECPB system during this period, had been available in sources such as the applicable Association of American Railroad (AAR) standard (S-4200). However, BHP personnel only appeared to recognise the potential significance of this designed characteristic following the March 2017 event.
After consultation with the equipment supplier, BHP personnel concluded that an added procedural control was essential in such situations to prevent the potential for a runaway to occur (that is, moving the automatic brake handle to the pneumatic emergency position to dump the brake pipe air). This learning resulted in BHP publishing OI 17-11 on 5 April 2017. The instruction contained the new requirement for drivers to secure all portions of the train by placing the automatic brake handle in the pneumatic emergency position, in conjunction with fully applying the independent brake and then manually applying handbrakes.
In effect, BHP was now aware of another ‘cause’ that could lead to an uncontrolled or uncommanded train movement, and it had also identified a critical preventative control to reduce the risk. Although published in OI 17-11 however, this critical control was not subsequently incorporated into the risk assessment for an RME interaction incident. Consequently, the effectiveness of the control was not evaluated as part of BHP’s normal risk management processes, including a control design assessment (CDA) and the verification activities associated with a material risk control assessment (MRCA).
Following OI 17-11 being published (5 April 2017), 2 scheduled MRCAs for the RME interaction incident risk event occurred in September 2017 and February 2018. Although records showed the assessment of various factors, there was no record showing consideration of the effectiveness of OI 17-11 in managing the associated risk during these activities.
A systematic assessment of the critical control associated with the procedure in OI 17-11 would have provided an opportunity to review relevant ECPB-related failures, and the associated BHP investigations and/or event recorder data, to determine the degree of driver compliance and overall understanding of the criticality of complying with the procedural controls in the instruction. However, such an assessment was not done at the time.
Following the runaway of M02712, BHP selected and reviewed 14 related events (including that involving M02712), finding 9 occasions where the driver had not followed the procedure and placed the automatic brake handle in the pneumatic emergency position. This included 6 events at locations with the potential for a runaway. By not examining these previous 13 events in a systematic manner at the time, a significant opportunity was missed to identify problems with the application of the critical control and take action to ensure it was being implemented more effectively prior to the 5 November 2018 runaway occurrence.
Summary
Although BHP’s risk assessment for an RME interaction incident identified numerous causes and critical controls for such an incident, it was broad in scope and had limited focus on the causes and critical controls for a train runaway event. In addition, it did not include the procedure for responding to brake pipe emergencies and penalties as a critical control, and its MRCAs did not test the effectiveness of this procedural control for preventing an uncommanded movement of a train on the main line.
The March 2017 event provided a valuable opportunity for BHP to identify the vulnerability to a runaway event associated with the ECPB overlay system’s integration with other systems. BHP used this opportunity to identify a missing critical (procedural) control and implement that control. However, it had not ensured that the control was effectively implemented (see also next section).
In addition, BHP did not effectively use the opportunity from the March 2017 to revisit the ECPB system’s integration with related systems to assure itself that it had effectively controlled the risk of a runaway event involving an ECPB train. The use of conventional risk assessment processes to identify problems in the integration between complex systems would probably been of limited effectiveness for this purpose. However, using a systems engineering approach (and methods) at this point would have increased the likelihood for identifying situations where the ATP system would be an ineffective control for stopping a train that had commenced rolling away following an ECPB emergency braking application.
Prior to the March 2017 event, the potential of a risk assessment using a conventional risk assessment approach to identify problems with the integration between complex systems would probably also have been limited, particularly when the risk assessment for RME interaction incident was so broad in nature. The use of a systems engineering approach (and methods) as part of the risk assessment process, particularly if the risk assessment was more focussed on the conditions or sequence of conditions that may cause runaway events, may have increased the potential to identify system limitations during that period.
Design and introduction of procedures for responding to brake pipe emergencies
Overview
In addition to the procedure for responding to brake pipe emergencies and penalties not being included in the risk assessment for an RME interaction incident, there were other problems with the way OI 17-11 was designed and introduced in April 2017. These included the application of processes for making the change, the resulting format of the change or design of the instruction, and the way the change was communicated to drivers. In addition, there were problems with the frequency of procedural changes and the design of the overall task of responding to brake pipe emergencies and penalties.
Application of processes for changing operating instructions
When issuing an operating instruction that made changes related to BHP’s rail rule book, the initiator was required to involve stakeholders, undertake a risk assessment and apply the management of change (MoC) procedure. For complex changes, relevant stakeholders were also required to be provided with an information package about the change.
BHP acknowledged a new rule implemented via an operating instruction would typically involve these processes, but it had allowed flexibility in the adoption of the processes, dependent on the extent or intent of the change. A determination not to implement the required processes would probably be reasonable when undertaking minor wording changes to an existing process to improve clarity.
As noted in the previous section, OI 17-11 introduced a new action requiring drivers of an ECPB train who experienced a loss of trainline communications with the EOTM displaying ‘off’ to move the automatic brake handle to the pneumatic emergency position. Although the change involved amending a single procedural step (or dot point), it introduced a safety-critical action, and therefore should not have been considered as a minor change.
However, BHP were unable to provide documentation that showed a risk assessment or change management process. Similarly, BHP were unable to provide a record of a determination of the extent or intent of the change that led to a decision not to adopt the required processes. A similar situation had also occurred with the introduction of OI 16-16, which replaced the applicable rule book procedure in full to include ECPB trains.
Design or format of the operating instruction
With regard to the resulting format of the change introduced in OI 17-11, and the way the change was communicated within the instruction:
No note with warning or caution information (using an appropriate symbol) was included in the instruction to indicate the importance of the new action, or the criticality of the consequences if the action was not completed. Such information can help focus attention on, and ensure better recall of, a procedural change.
No information was provided in the instruction regarding the reason for the change. Explaining why a change has been made is widely regarded as an essential part of implementing procedural changes (Barshi and others 2016, Degani and Weiner 1994). In addition, having a detailed mental model or understanding of a system facilitates better performance in a range of situations (Wickens and others 2015), including better recall of procedures (Kieras and Bovair 1984). For example, including information about the CCDs’ 60-minute shut-down feature would have highlighted the feature and enabled drivers to incorporate this information into their mental model of how the system worked and needed to be managed when responding to emergency situations.
The amended procedural step was presented in a red font. Although this colour increased the text’s potential salience relative to other procedural steps, the use of red font in this way was not consistent with how changes were indicated in previous operating instructions (normally in yellow highlight), and it was not consistent with how important procedural steps were indicated in previous operating instructions (normally in bold). Red font was also used elsewhere in the instruction for other purposes.
The amended procedural step included 3 actions: moving the automatic brake handle to the pneumatic emergency position, fully applying the independent brake, and manually applying the required handbrakes. The second 2 actions had not changed from previous versions of the instruction, yet they were also presented in red font; this reduced the potential salience of the new required action.
Although all 3 actions were associated with securing a train, they were distinct actions and could have been more usefully presented as separate procedural steps (or dot points). Guidelines for writing procedures include keeping sentences short and presenting each action on a separate line (for example, Barshi and others 2016).
In summary, although the new action in the procedure was presented in red font, it was not necessarily salient, and the instruction did not clearly state the importance of the new action and the reasons why it was introduced or important.
Other communication processes
In addition to the dissemination of the operating instruction itself, BHP communicated information about changes to procedures or instructions through supervisors providing safe start briefings. The extent to which the change introduced with OI 17-11 was discussed in safe start briefings was not able to be determined as BHP was unable to provide a copy of the relevant briefing sheet. However, based on the nature of the information in the safe start briefing sheets following other changes to the operating instruction (such as OI 17-09 or OI 18-72), it is unlikely there was any information provided to drivers that was not contained in the instruction itself.
Although it was a supervisor’s responsibility to ensure drivers receiving the information in a safe start briefing understood its content, the practice of disseminating information through the briefings did not address how the supervisor would satisfy themselves of the drivers’ level of understanding of the information, other than obtaining a signature from the recipients acknowledging receipt of the information.
As well as safe start briefings, BHP also conducted driver reaccreditation training and assessment at regular intervals. This provided an opportunity to undertake structured training and assessment of drivers’ understanding of operational rules and procedures that had changed during the accreditation period. The most recent reaccreditation assessment undertaken by the driver of M02712 (in August 2018) included questions related to ECPB operation and the required response to various fault conditions, including that associated with the EOT ‘off’. Although the driver answered that question correctly, it is noted that the question did not refer specifically to an emergency brake application or securing the train, and the nature of any associated classroom discussion on the topic prior to the assessment could not be determined.
Overall, it is unclear to what extent BHP’s drivers were provided with information about the procedural change introduced in OI 17-11 other than what was in the instruction itself. However, the fact that 9 of 14 drivers who were required to use the procedure did not move the automatic brake handle to the pneumatic emergency position is consistent with many drivers not being provided with clear and relevant information about the importance and reasons for the change.
Frequency of procedural changes
BHP used operating instructions to communicate new and amended procedures to drivers during and following the commencement of ECPB operations rather than update the relevant module in the rail rule book. Overall, 13 separate instructions relating to brake pipe emergencies and penalties were issued in the 3 years between February 2014 and April 2017, with a further revision issued with OI 18-72 just prior to the 5 November 2018 occurrence. The only operating instruction that was subjected to assessment under the BHP management of change procedures was OI 17-09.
Frequent procedural changes can increase memory demands when trying to correctly recall the latest version and not confuse some of the steps with previous versions. When under stress, people can also revert to previously learned skills or versions of a procedure (Barshi and others 2016). In addition, frequent changes to procedures can lead to personnel believing their operator’s system is unstable, which may diminish the importance they attribute to any changes (Degani and Weiner 1994).
Design of the task for responding to brake pipe emergencies
The overall design of the task of responding to brake pipe emergencies and penalties also placed significant memory demands on a driver and increased the likelihood of them not correctly remembering all of the required procedural steps. More specifically:
The task required drivers to remember a different set of procedural steps for several different situations – depending on whether it was an ECPB or pneumatic train, whether it was a brake pipe emergency or a penalty, the extent of the train brake command (TBC), and whether the train was moving or at a stand. Although drivers were required to familiarise themselves with the procedure (and any changes as they were issued), they were not required to carry a copy with them and a copy was not provided on each train.[42] Referring to a procedure, checklist or job aid is a very well-known and reliable method for minimising errors of omission, particularly for rarely-performed tasks that do not need an immediate response (Wisher and others 1999, Sanli and Carnahan 2018, Barshi and others 2016).
Drivers were rarely required to conduct the task while operating a train and they were not provided with opportunities to practice the task. The term ‘skill decay’ (or skill fade) refers to the loss of trained or acquired skills or knowledge following periods of non-use (Arthur and others 1998). Skill decay increases as the retention interval (or time since learning) increases, and it also increases depending on the quantity and quality of the initial and recurrent training and the amount of on-the-job exposure (Arthur and others 1998, Sanli and others 2018, Vlasblom and others 2020).
The procedure required a set of discrete actions, many of which did not provide meaningful cues or feedback to prompt the next action in the procedure (including the application of the automatic brake). This type of procedural task is more likely to be associated with skill decay than many other types of tasks (Goodwin 2006, Sothard and Nicholson 2001, Wisher and others 1999).
There was no process in place to cross-check the performance of a driver who was conducting the task. Research has shown that errors of omission are often difficult to detect by the people who make them (Sarter and Harrison 2000). In safety-critical systems, human error will occur and such systems need processes in place to not only reduce the likelihood of errors but also detect and recover from them. This would ideally involve some form of engineered risk control providing feedback to indicate that an important action had not been completed. Alternatively, the task could have been designed to require another person to cross-check a driver’s performance. In the case of responding to a brake pipe emergency, the train controller was the person best placed to ensure specific actions were completed, as they were already involved in communications with the driver and determining the number of handbrakes required.
Summary
OI 17-11 introduced a simple yet safety-critical action for drivers to complete in the event of a loss of trainline communications resulting in the EOTM displaying ‘off’. However, BHP did not follow its defined processes for making the procedural change, and the extent to which the resulting OI 17-11 was reviewed or assessed prior to being issued was not able to be determined.
Overall, the task of responding to brake pipe emergencies and penalties relied extensively on a driver’s memory, with limited processes in place to facilitate or cross-check a driver’s performance to ensure all safety-critical actions were completed. In addition, although OI 17-11 (and subsequently OI 18-72) contained a safety-critical action (to apply the automatic brake handle to the pneumatic emergency position), BHP did not clearly communicate the importance and reasons for this action to drivers, reducing the potential for drivers to correctly recall the action, particularly given the context of many previous changes to operating instructions and the low frequency with which the task was required to be conducted.
Loss of trainline communications on M02712
As previously noted, a loss of trainline communications resulting in the EOTM displaying ‘off’ was a fault condition (in combination with other events and conditions) that could result in a runaway event involving an ECPB train. A break in the trainline could occur due to a variety of reasons at any connection point along the train and at any time.
In the case of M02712, recorded information showed that the loss of trainline communications occurred at 0338 as the train was approaching Garden. The communications loss was due to a problem with one of the inter-car connectors towards the front of the train. The driver recalled seeing a disconnected inter-car connector at about ore car 10 in the first rake. Train M02712 had primarily NYAB (square type) connectors installed but it also had 12 WABTEC (round type) connectors installed, which were under trial on ore cars 2 to 8 in the first rake. The driver’s description of the connector, and its relative location in the first rake of ore cars, was consistent with the disconnection involving a WABTEC type connector.
Based on the available information, the exact mechanism that led to the disconnection could not be determined. The driver’s description was consistent with the connector simply disconnecting. The event occurred as the train passed over an access road level crossing, and it is possible that the connector was hanging too low after the ore car was loaded, and it contacted infrastructure at the level crossing, resulting in the disconnection.
Consistent with the ECPB system’s design, the loss of trainline communications triggered a 120% train brake command (TBC) emergency brake application, applied the emergency brake interlock, and disconnected the trainline power feed. The automatic brake was then held by the interlock but only for a limited time period. For the ore cars rear of the point of break (in this case most of the train), the CCDs shut down after 60 minutes and released their brake application.
The loss of trainline communications occurred at Garden South and the track gradient was -1.5%, the steepest gradient of the section between Yandi Junction and Nelson Point. This meant that to secure the train using handbrakes, which was required before commencing work to repair the train, the handbrakes on all 268 ore cars had to be applied, which would take much longer than 60 minutes. Consequently, unless the driver moved the automatic brake handle to the pneumatic emergency position, the train would commence rolling away.
In summary, while train M02712 was approaching Garden and on a descending grade, one of 12 inter-car connectors undergoing a trial near the front of the train disconnected. This caused a loss of trainline communications and an emergency brake application. It also resulted in the CCD on each of the ore cars rear of the break in the trainline initiating an in-built 60-minute shut-down feature.
In addition to limitations related to the introduction of ECPB overall, there were also related limitations with the introduction of the trial connectors. BHP managed the introduction of the WABTEC type connectors in 2018 through its management of change process, with the trial commencing on 1 November 2018. The management of change process included a risk assessment, which considered the potential for adverse operational outcomes from the failure of a trial connector. This was likely in recognition of the unknown performance characteristics of the connector when fitted to the BHP ore cars at that time. The forecast consequence from the assessment was principally a financial loss arising from main line service interruptions while recovering from the failure.
The risk assessment for the trial did not consider other potential operational risk factors, such as an RME interaction incident (or train runaway), that could potentially arise from a loss of trainline communications. It also did not consider the relative risk of a failed connector at the front of the train compared to at the rear of the train.
It is likely that locating the ore cars with the trial connectors toward the front of the train was either to provide ready access for the driver to repair a connector fault if it occurred (after securing the train), or as a consequence of shunting the 7 ore cars into a 134 ore car rake and then marshalling the rake for introduction into the train service. Regardless of the reason, the location of the trial inter-car connectors at the front of train M02712 introduced an unrecognised hazard, as it meant that if there was a problem with one of the connectors almost all of the train would not be in communication with the HEU and the CCDs would shut down after 60 minutes. Overall, the absence of a documented consideration of this issue in the trial’s risk assessment was consistent with BHP personnel still not fully understanding the limitations and constraints associated with the integration of the ECPB system with other systems.
Driver response to the loss of trainline communications
In response to an ECPB emergency brake application when moving, with the EOTM displaying ‘off’, the applicable operating instruction required a driver to follow a number of procedural steps prior to commencing an inspection to identify and rectify the fault condition. From April 2017 (in OI 17-11 and subsequently in OI 18-82), these steps included:
declare an emergency
advise train control of relevant details (including location)
request protection of adjacent tracks
secure all portions of the train by placing the automatic brake handle in the pneumatic emergency position, applying the independent (locomotive) brakes, and manually applying handbrakes as confirmed by train control.
On the 5 November 2018, the driver of M02712 contacted train control to declare an emergency and provided relevant details. The driver also applied the locomotive independent brake. The controller placed blocks of the adjacent track for protection, and the controller and driver then had discussions to determine the exact location of the train and therefore the amount of handbrakes required, after which the driver centred the reverser and turned the generator field off before leaving the locomotive to commence applying the handbrakes to the first rake of ore cars.
As already noted, a critically-important action for responding to this type of fault condition was to move the automatic brake handle to the pneumatic emergency position (to dump the brake pipe air). As evidenced by the recorded data, the driver omitted this action, and therefore the critical control in the procedure was not implemented. Accordingly, following the CCDs on most of the ore cars shutting down after 60 minutes (consistent with the way they were designed) and insufficient handbrakes having been applied at that time, the train commenced rolling away on the descending grade.
In effect, the driver omitted one action in one step in the procedure. Omitting a step or an action is one of the most common forms of human error (Reason 2002), and it can occur due to a wide variety of reasons. In this case the omission was likely associated with the inherent limitations of long-term memory, the driver’s understanding of the relevant systems and the way the procedural change introduced in OI 17-11 was communicated to drivers.
More specifically, although the driver had received OI 17-11 and OI 18-72, and was required to familiarise themselves with the contents, the driver had not had to apply the procedure for this type of emergency fault condition before and had no previous opportunity to practice the required response, increasing the likelihood of skill (or knowledge) decay. The driver also did not have a copy of the procedure to refer to when doing the task, and the instruction itself did not clearly indicate the importance of the action or the reasons why it was introduced, reducing the potential for correct recall. In addition, the driver stated they were not made aware of the ECPB system’s 60-minute shut-down feature. As previously discussed, the frequency of procedural changes and the overall design of the task also placed significant memory demands on a driver.
The ATSB considered a range of other explanations for the driver’s omission, including fatigue (see Driver fatigue), time pressure and distraction. Applying the automatic brake at the same time as the locomotive brake would be consistent with the design of the procedure (that is, both actions were contained in the same procedural step). At that time, it is unlikely that time pressure or distraction was a factor. However, the driver also had subsequent opportunities to place the automatic brake handle into the pneumatic emergency position when waiting for advice from train control and when completing the tasks within the three-step protection process. During that period, the driver’s attention may have been distracted by communications with train control and the additional task of obtaining an exact marker for the train’s position. It is also possible the driver was experiencing a degree of perceived time pressure to start resolving the situation. However, without having the appropriate knowledge of the 60-minute shut-down feature, it is unclear whether the driver spending additional time reviewing the situation and their actions would have led to them identifying the need to move the automatic brake handle to the emergency position.
In summary, when applying the emergency procedure for responding to a loss of trainline communications with the end of train monitor displaying ‘off’, the driver did not place the automatic brake handle into the pneumatic emergency position, which would have vented the brake pipe pressure to zero and applied the train brakes via the pneumatic system in addition to the system-initiated ECPB application.
The driver was conducting the procedure from memory, and their memory of the procedure could have decayed or been affected due to a range of factors associated with the way the overall task was designed and the way the change in the operating instruction was designed and introduced. As noted in Design of the task for responding to brake pipe emergencies, for rarely-performed tasks that do not need an immediate response, being provided with a copy of the procedure or a checklist and being required to use it, or having another person review their actions, would have been effective ways of ensuring that the driver did not omit any critical actions when performing the procedure.
Recovery controls – integration between ATP and ECPB
After M02712 commenced rolling away at 0440, without the driver on board, there was no recovery control available to stop the train in this specific situation, other than forcing a derailment.
As already noted, BHP’s trains were fitted with an automatic train protection (ATP) system. This system was designed to apply penalty braking to a train in certain conditions, such as if there was an uncontrolled / uncommanded roll away of the train. More specifically, when a locomotive was stationary with its reverser in the neutral (centre) position and the ATP detected a movement of more than 0.5 m, the ATP requested a penalty braking application to stop the rollaway. In addition, If the locomotive exceeded the target speed limit, alarms would sound to prompt the driver to reduce speed. If this did not occur (such as if there was no driver on board), the ATP system automatically communicated with the braking system to request a penalty brake application to stop the train.
For train M02712, the ATP system functioned as designed, and it made the appropriate requests for penalty braking after it started rolling away and then when it exceeded the relevant speed limits. However, given the situation that existed for M02712 on 5 November 2018, these requests from the ATP system were ineffective in initiating any braking.
As discussed earlier, this was associated with the way the ECPB system was integrated with the ATP system. In the event the CCDs shut down, and a train was not effectively secured against unintended movement, any subsequent ATP penalty commands to the ECPB system would be ineffective in stopping the uncommanded movement. In other words, the ATP system and the ECPB system on BHP’s trains could not interface to dump brake pipe pressure if an ECPB emergency or penalty brake application became ineffective in arresting an uncommanded train movement.
BHP had likely weighted the ATP system heavily as an engineered control that would prevent or recover events where a train moved uncommanded, travelled at excessive speed or outside the limit of authority. However, as discussed in earlier sections, BHP had not identified the limitation with the integration between the ATP and ECPB systems during its implementation of the ECPB overlay system, and its investigation into similar events involving drivers response to a 120% brake pipe emergency with EOT off, or in association with subsequent risk assessments for an RME interaction incident.
Three-step process for accessing rail-mounted equipment
BHP’s three-step protection process was required to be applied before conducting work on any train, and it provided another control against an uncommanded or uncontrolled train movement. The driver of M02712 applied the three-step protection tasks shortly after the train stopped at Garden, initially applying the independent brake, then later placing the reverser to the neutral position and opening the generator switch.
The instructions in the Rail Rule Book module 1 included an additional requirement for the driver to apply the train brake (automatic brake) where the gradient of the track and/or weight of the train may allow the vehicles to move. However, this requirement likely related to the context of securing the train when undertaking tasks such as train crew changeover or rolling stock inspection, and not specifically in response to a brake pipe emergency and penalty (where the applicable procedure was published separately in a series of operating instructions). In addition, in the case of M02712, the driver was aware that the ECPB interlock was already applying a 120% TBC or maximum braking effort.
The three-step protection process not only involved the driver, but also placed responsibilities on the workers seeking to access the rolling stock profile. In this case, the responsible person for the worksite was required to contact the train driver to request and confirm the application of three-step protection before entering the rolling stock profile to apply handbrakes. The Redmont maintenance gang arrived at the 210 km mark and the driver of M02712 was some distance away, already applying handbrakes near the front of the train. There were radio communications between the controller and the maintenance gang, and communications between the controller and the driver, and the controller had advised the gang and the driver to contact each other. However, at no stage did the gang directly communicate with the driver to confirm the application of three-step protection.
On arrival, the gang mistook the rear of an unloaded train (M02727), which was stopped on the eastern track at Garden South, to be the loaded train M02712, which was stopped further south on the western track, and the gang started applying the handbrakes on the wrong train. Ultimately, this error did not contribute to the runaway of M02712, as the gang would not have had time to apply sufficient handbrakes prior to the end of the 60-minute period after the loss of trainline communications. Nevertheless, applying handbrakes to the incorrect train increased the risk of injury to personnel working on the rolling stock.
The train controller tasked the Redmont gang to attend M02712 at Garden South. There is no record of the train controller advising the gang of the later arrival of the second train at Garden south (M02727), which was coincidentally stopped at that location due to the protections placed by the controller. Therefore, it is likely the gang were unaware that a second train was at Garden south. However, direct communication between the gang and driver in relation to the application of three-step protection and arrangements for the application of handbrakes would have provided an opportunity to identify the error made by the gang.
Emergency response – interface coordination
Following the notification by the driver of M02712 of the emergency braking event, the train controller coordinated the internal arrangements for protecting the train, dispatching other support personnel and communicating with the driver at 10-minute intervals. After notification by drivers that M02712 had run away, the controller continued to monitor the welfare of the driver and support personnel. In addition, the controller applied additional protections in an attempt to stop the movement of train M02712, and alerted the drivers of other trains ahead of the evolving situation before instructing them to stop and move to a position of safety.
In general, the train controller’s actions were consistent with the requirements of the rules and procedures for responding to an emergency of this type. The ensuing internal communications between the controller and field personnel supported the safety of BHP operational personnel who were or had the potential to be affected.
In addition to communicating internally within BHP’s operations, there was also a requirement to communicate with external parties. The Newman to Port Hedland railway had several locations where an interface occurred between BHP and an adjoining rail infrastructure manager (RIM), pastoralist/local government, or a manager of a public or private road. The associated interface agreement with each party detailed the responsibilities for the notification of accidents, incidents or near misses that occurred which had the potential to increase risk to either party at the interface. However, the agreements did not include requirements related to notifying the other parties, including other RIMs’ train control, of an event in progress on the network that had the potential to increase risk at an interface, such as a runaway.
Train M02712 travelled uncontrolled and at speed for about 91 km, traversing several locations where BHP’s railway interfaced with other parties. Two of these locations interfaced with another RIM, where a similar heavy-haul railway passed over the BHP’s railway via bridge infrastructure. BHP risk assessments identified the potential for damage to rail overpasses arising from the derailment of a BHP train due to an overspeed. Neither the train controller or a BHP supervisor or superintendent contacted the parties that interfaced with the section of the railway affected by the runaway of train M02712. Had they been informed, the other RIMs’ train control could have ensured that ensured that rail vehicles and personnel on their networks were positioned away from potential risk associated with the runaway train.
Fatigue and fatigue management
Driver fatigue
Determining whether a person is experiencing a level of fatigue that is likely to adversely influence performance involves considering the amount of recent sleep and work and a range of other factors. With regard to M02712:
The driver was conducting the sixth of 7 12-hour night shifts from 2200–1000. They recalled getting about 4–5 hours sleep following each night shift, and overall about 5–6 hours sleep in total during each break between night shifts. This appeared to be consistent with, or slightly more than, what would be expected based on the available research for this type of shift (see Appendix B). Most people need at least 7–8 hours of sleep each day to achieve maximum levels of alertness and performance, and research has shown that restricting sleep to 6 hours or less a night over several nights will result in significant performance decrements (Banks and Dinges 2007, Watson and others 2015).
The train stopped at Garden at about 0340 in the morning, which is during the window of circadian low (0200–0600) for a person with a normal sleep-wake cycle. Although the driver was completing their sixth night shift, it is unlikely that their sleep-wake cycle had significantly adapted during this period. People exposed to a significant amount of sunlight each morning are unlikely to shift their sleep-wake cycle (Smith and Eastman 2012), and in this case sunrise in the Yandi to Port Hedland area was occurring at about 0715 in the morning and the driver was probably getting to bed at about 1100 following each night shift.
The driver had risen early (about 0230 in their established sleep-wake cycle) to travel from Adelaide to the Yandi camp prior to their first night shift. Although they were provided with 5 hours opportunity to settle in upon arrival at the camp, it is unlikely that they would have obtained sufficient sleep to overcome their initial sleep debt prior to the first night shift.
The driver reported that they found a roster pattern with night shifts commencing at about 2200 the most difficult for obtaining sleep. This pattern equated to a start time of 2330–0030 in their established sleep-wake cycle when they first arrived at the camp.
The driver reported that they were ‘a little bit tired’, but research indicates that people will generally underestimate their level of fatigue (Battelle Memorial Institute 1998). In addition, people underestimate the impact of several days of sleep restriction (Banks and Dinges 2007, Watson and others 2015).
The driver was conducting a single-driver operation, and would normally expect to have a 30-minute rest break after about 6 hours. On the day of the accident, the rest break had not yet occurred and would not have occurred until about 0500 as there was a delay before train M02712 became available.
Biomathematical modelling, using 3 different models, indicated that an average person working the driver’s roster pattern would have a significant level of fatigue during the last 3 of the 7 night shifts. All models have assumptions and limitations and use different inputs, and they do not consider all the factors that can influence fatigue. However, in this case it is noteworthy that the indicated levels of fatigue were significant and consistent across the 3 models.
In a FIFO work environment, workers generally have less domestic demands and distractions and need to conduct fewer non-work tasks. In this case, it is also unlikely the driver’s sleep was disrupted by the quality of the accommodation or other local factors. In addition, the driver only had short commute times between the accommodation and signing on for work. Although these contextual factors will have facilitated the driver’s ability to utilise their available sleep opportunities, they will not have been sufficient for most people to overcome the fundamental problems associated with restricted sleep and related factors, such as the time of day.
In summary, due to cumulative sleep restriction over several days of night shifts, the time of day (0340) and other factors, the driver was probably experiencing a level of fatigue known to adversely influence performance when the train came to a stop at Garden.
Fatigue can have a wide range of impacts on human performance, and the driver’s level of fatigue would have increased the likelihood of making errors when performing many types of tasks, including those involving the use of working memory (Goel and others 2009). However, although some research has shown that fatigue can increase errors associated with retrieval from long-term memory, the results are inconsistent (Alhoha and Polo-Kantola 2007).
Overall, the extent to which fatigue contributed to the driver’s error of not moving the automatic brake handle to the pneumatic emergency position could not be determined. As previously discussed, that error was likely related to the driver’s understanding of the relevant systems and the manner in which the operating instructions were presented. It is also noted that a number of other drivers had made a similar error, and the extent to which they may have been experiencing fatigue was unknown. In other words, although fatigue increased the likelihood of making an error, it is unclear from the evidence available whether the driver’s error would still have occurred on this occasion if the driver had been experiencing a lower level of fatigue.
Management of rosters and fatigue risk
BHP's fatigue management processes required its train drivers to be rostered on 7 12-hour shifts, followed by a 24-hour break and then 7 12-hour shifts, with the roster pattern commencing at all times of the day. As previously noted, research has shown that roll-over roster patterns or swings that include 7 consecutive 12-hour night shifts present an elevated risk of fatigue, particularly in environments where it is unlikely that workers will adapt their sleep-wake cycles. This risk is further exacerbated depending on the timing of the shifts, with night shifts ending in the late morning likely to lead to the least amount of sleep.
Given this fatigue risk potential, BHP needed to have processes in place to provide assurance that its drivers could actually obtain sufficient sleep between shifts (as well as obtain sufficient rest during shifts) to maintain adequate levels of alertness when carrying out their safety-critical tasks. BHP had recognised that its roll-over roster patterns for WAIO train drivers were problematic and, even with a set of control measures, assessed the associated risk level was ‘high’ (compared to a raw risk level before treatment of ‘extreme’).
Some of the specified control measures were clearly important for minimising potential fatigue-related problems (such as providing suitable air-conditioned accommodation and a 4-hour sleep opportunity prior to commencing the first night shift, as well as ensuring trains were fitted with ATP for single-driver operations). However, given the realistic potential for many drivers to not be able to obtain sufficient sleep on some of their roster patterns (depending on their start time), there needed to be robust processes in place to ensure that sufficient sleep was actually being obtained.
The primary control in place to assess the amount of sleep being obtained by drivers was the fatigue assessment tool. This short questionnaire was required to be completed by a driver twice each swing, and as required if a driver felt fatigued. However:
Although this tool contained relevant questions, there were limitations with how the overall score was derived. A driver could have an overall low-risk score yet one of the sleep criteria (such as sleep in the last 24 hours or sleep in the last 48 hours) could have been significantly affected; they could also have an overall medium-risk score but have high risk associated with multiple criteria.
Drivers were required to complete the tool prior to the second shift and prior to the ninth shift of each swing. Completing it prior to the second shift would include the last sleep prior to commuting to the camp, sleep after arriving at the camp and sleep after the first shift, and completing it after the ninth shift would include the sleep during the 24-hour recovery break at the end of the first week and sleep after the next shift. Depending on the initial sign-on time, such scores would often be better than if they were completed after multiple night shifts.
Concerns about self-reporting fatigue are commonly perceived amongst train crew in the rail industry (for example, Fitness and Naweed 2017), and anecdotal evidence indicated that BHP’s drivers may also have been unlikely to complete the fatigue assessment tool accurately or report being fatigued. In addition, the overall proportion of BHP’s completed fatigue assessments that resulted in low-risk scores (97.6%) was unrealistic, and much higher than would be expected given the results of research associated with night shifts and roll-over roster patterns.
One of the key questions on the tool was a self-rating of alertness; as already noted, people will generally underestimate their level of fatigue or the influence of cumulative sleep restriction.
The importance of providing realistic answers for each of the questions could have been reinforced through detailed fatigue awareness training. However, recent training material did not emphasise the amount of sleep required each day to maximise alertness and performance.
BHP had not conducted any sleep studies, surveys or similar research to determine how much sleep its train drivers were actually obtaining during swings starting at different times of day.
BHP had applied the FAID biomathematical model of fatigue (BMMF) to some of its FIFO train driver swings. Although these applications had indicated the swings had very high scores (related to most normal work rosters), and scores that significantly exceeded the nominated fatigue tolerance threshold, this did not appear to generate any additional controls or treatments. The results were compared relative to other swings that had very high scores rather than treated as potential problems that needed further assessment.
BHP had conducted a limited number of critical control observations and audits, but these were very limited in scope. It had not conducted any independent or detailed reviews of its train driver rosters or fatigue management system until August 2018. That review, and a related review in February 2020, outlined a range of potential concerns and recommendations.
In summary, BHP's fatigue management processes required its train drivers to be rostered on 7 12-hour shifts, followed by a 24-hour break and then 7 12-hour shifts, with the roster pattern commencing at a wide variety of times of day. Such roster patterns were conducive to result in cumulative sleep restriction and levels of fatigue likely to adversely influence performance on a significant proportion of occasions, and BHP had limited processes in place to ensure that drivers actually obtained sufficient sleep when working these roster patterns.
ATSB investigation report findings focus on safety factors (that is, events and conditions that increase risk). Safety factors include ‘contributing factors’ and ‘other factors that increased risk’ (that is, factors that did not meet the definition of a contributing factor for this occurrence but were still considered important to include in the report for the purpose of increasing awareness and enhancing safety). In addition ‘other findings’ may be included to provide important information about topics other than safety factors.
Safety issues are highlighted in bold to emphasise their importance. A safety issue is a safety factor that (a) can reasonably be regarded as having the potential to adversely affect the safety of future operations, and (b) is a characteristic of an organisation or a system, rather than a characteristic of a specific individual, or characteristic of an operating environment at a specific point in time.
These findings should not be read as apportioning blame or liability to any particular organisation or individual.
From the evidence available, the following findings are made with respect to the runaway and derailment of loaded ore train M02712 that occurred on the 5 November 2018 near the 211 km mark south of Port Hedland, Western Australia.
Contributing factors
BHP predominately managed the implementation of its electrically controlled pneumatic brake (ECPB) overlay and modification of automatic train protection (ATP) systems in 2011–2015 at an individual system level rather than through the application of a structured engineering approach. In the absence of a systems engineering framework, BHP did not identify and manage significant characteristics of how the ECPB, ATP and conventional pneumatic braking systems interacted in response to certain fault conditions.
Although BHP’s risk assessment for a rail-mounted equipment interaction incident identified numerous causes and critical controls for such an incident, it was broad in scope and had limited focus on the causes and critical controls for a train runaway event. In addition, the risk assessment did not include the procedure for responding to brake pipe emergencies and penalties as a critical control and BHP’s material risk control assessments (MRCAs) did not test the effectiveness of this procedural control for preventing an uncommanded movement of a train during main line operations. (Safety issue)
The task of responding to brake pipe emergencies or penalties relied extensively on a driver’s memory, with limited processes in place to facilitate or cross-check a driver’s performance to ensure all safety-critical actions were completed. (Safety issue)
Although operating instructions OI 17-11 (5 April 2017) and then OI 18-72 (3 November 2018) contained a safety-critical action (to apply the automatic brake handle to the pneumatic emergency position), BHP did not clearly communicate the importance and reasons for the safety-critical action to drivers, reducing the potential for the drivers to correctly recall this procedural action. (Safety issue)
Approaching Garden and on a descending grade, one of 12 inter-car connectors undergoing a trial near the front of the train disconnected. This caused a loss of trainline communications, resulting in the car control device (CCD) on each of the ore cars rear of the break in the trainline initiating an in-built 60-minute shut-down feature.
During implementation of the emergency procedure for responding to a loss of trainline communications with the end of train monitor displaying ‘off’, the driver did not place the automatic brake handle into the pneumatic emergency position, which would have vented the brake pipe pressure to zero and applied the train brakes via the pneumatic system.
The automatic train protection (ATP) and electronically controlled pneumatic braking (ECPB) systems on BHP’s trains could not interface to dump brake pipe pressure if an ECPB emergency or penalty brake application became ineffective in arresting an uncommanded train movement. (Safety issue)
Other factors that increased risk
On arrival at the 210 km mark, the Redmont maintenance gang did not directly communicate with the driver of M02712 and did not confirm the driver had implemented the BHP three-step process. The gang subsequently started applying handbrakes to another train (M02727), without three-step protection being applied on that train, which increased the risk of injury to personnel working on the rolling stock.
BHP’s emergency response procedures did not ensure rail infrastructure managers that interfaced with BHP's rail network were alerted to the emergency that could affect safety at the interface.
Due to cumulative sleep restriction over several days of night shifts, the time of day (0340) and other factors, the driver of MO2712 was probably experiencing a level of fatigue known to adversely influence performance.
BHP's fatigue management processes required its train drivers to be rostered on 7 12‑hour shifts, followed by a 24-hour break and then 7 12-hour shifts, with the roster pattern commencing at a wide variety of times of day. Such roster patterns were conducive to result in cumulative sleep restriction and levels of fatigue likely to adversely influence performance on a significant proportion of occasions, and BHP had limited processes in place to ensure that drivers actually obtained sufficient sleep when working these roster patterns. (Safety issue)
Safety issues and actions
Central to the ATSB’s investigation of transport safety matters is the early identification of safety issues. The ATSB expects relevant organisations will address all safety issues an investigation identifies.
Depending on the level of risk of a safety issue, the extent of corrective action taken by the relevant organisation(s), or the desirability of directing a broad safety message to rail industry, the ATSB may issue a formal safety recommendation or safety advisory notice as part of the final report.
All of the directly involved parties were provided with a draft report and invited to provide submissions. As part of that process, each organisation was asked to communicate what safety actions, if any, they had carried out or were planning to carry out in relation to each safety issue relevant to their organisation.
Descriptions of each safety issue, and any associated safety recommendations, are detailed below. Click the link to read the full safety issue description, including the issue status and any safety action/s taken. Safety issues and actions are updated on this website when safety issue owners provide further information concerning the implementation of safety action.
Risk assessment for a rail-mounted equipment interaction incident
Safety issue description: Although BHP’s risk assessment for a rail-mounted equipment interaction incident identified numerous causes and critical controls for such an incident, it was broad in scope and had limited focus on the causes and critical controls for a train runaway event. In addition, the risk assessment did not include the procedure for responding to brake pipe emergencies and penalties as a critical control and BHP’s material risk control assessments (MRCAs) did not test the effectiveness of this procedural control for preventing an uncommanded movement of a train during main line operations.
Task design – brake pipe emergencies and penalties
Safety issue description: The task of responding to brake pipe emergencies or penalties relied extensively on a driver’s memory, with limited processes in place to facilitate or cross-check a driver’s performance to ensure all safety-critical actions were completed.
Operating instructions – brake pipe emergencies and penalties
Safety issue description: Although operating instructions OI 17-11 (5 April 2017) and then OI 18-72 (3 November 2018) contained a safety-critical action (to apply the automatic brake handle to the pneumatic emergency position), BHP did not clearly communicate the importance and reasons for the safety-critical action to drivers, reducing the potential for the drivers to correctly recall this procedural action.
Safety issue description: The automatic train protection (ATP) and electronically controlled pneumatic braking (ECPB) systems on BHP’s trains could not interface to dump brake pipe pressure if an ECPB emergency or penalty brake application became ineffective in arresting an uncommanded train movement.
Safety issue description: BHP's fatigue management processes required its train drivers to be rostered on 7 12-hour shifts, followed by a 24-hour break and then 7 12-hour shifts, with the roster pattern commencing at a wide variety of times of day. Such roster patterns were conducive to result in cumulative sleep restriction and levels of fatigue likely to adversely influence performance on a significant proportion of occasions, and BHP had limited processes in place to ensure that drivers actually obtained sufficient sleep when working these roster patterns.
Safety action not associated with an identified safety issue
Whether or not the ATSB identifies safety issues in the course of an investigation, relevant organisations may proactively initiate safety action in order to reduce their safety risk. The ATSB has been advised of the following proactive safety action in response to this occurrence.
Additional safety action by BHP
Following the runaway and derailment accident involving of M02712, BHP leadership teams of the involved parties met to find the ‘root cause’ of the process noncompliance. BHP subsequently implemented additional controls via the issue of operating notice 18-216[43] requiring that any person responding to an event was to meet the driver of the affected train in person upon arrival. The following day BHP published operating notice 18-219[44] superseding notice 18-216.
Operating notice 18-219 included added detail on the requirement for support personnel to liaise with both the train control and the locomotive driver to:
determine the location and lead locomotive of the train requiring assistance
establish communication with the train driver
confirm lead locomotive and location
meet the train driver at an agreed location
confirm three step protection is applied
work under the direction of the train driver.
The notice reinforced that accessing rolling stock had to be conducted per Rail Rule Book Module 1: General Rail Rules and Procedures (section P1-5.0 regarding three step protection).
Operating notice 18-219 expired on 11 April 2019.
On 31 January 2022, BHP updated the above proactive action, advising:
The requirement for a work group to meet with the lead locomotive was included in work instruction 0105931 from version 3.0 (Oct 2019).
In 2021, as part of BHP’s work instruction uplift project (which was aimed at simplifying content) this requirement was reviewed and deemed to not be practical because the driver would not always be at the lead locomotive of the train. The requirement was also obscure to all potential stakeholders that could render assistance in similar scenarios. Accordingly, the requirement was removed from work instruction 0105931, and a new requirement was inserted for a work group supervisor to be appointed to communicate directly with the driver and the work group.
BHP is currently updating and simplifying our Rail rule book to further improve on the safe operation of our rail network. This will include a new digital portal that is designed to make documentation relevant to the task more easily accessible to the end user.
Additional safety action by the Office of the National Rail Safety Regulator
On 20 November 2018, post the runaway of M02712, the Office of the National Rail Safety Regulator (ONRSR) issued safety alert number RSA-2018-002 to rail transport operators in relation to the use of electronically controlled pneumatic braking (ECPB) and automatic train protection (ATP) systems (Appendix C). The alert encouraged operators to:
conduct an assessment of the interaction between the ECP braking system and the mechanical pneumatic braking system following an unexpected (penalty) braking intervention on a train configured for ECP braking.
determine whether the ECP braking system is designed to the AAR S-4200 standard
determine whether the 60-minute release has been programmed within the ECP braking software
conduct a risk assessment on the use of ECP braking for the prevention of the event of a rollaway incident
conduct a risk assessment on the effectiveness of the ATP system in the event of an ECP braking system failure.
Following the issue of the safety alert, ONRSR identified rail transport operators that may use ECPB in their railway operations. The regulator liaised with identified operators and ensured those that were affected undertook the actions as outlined in the safety alert. ONRSR also liaised with industry vendors of ECPB systems to obtain assurance of corrective actions following the accident.
In March 2019, ONRSR published a safety message titled Importance of a System Engineering Approach, which stated:
Following recent incidents and observations the Office of the National Rail Safety Regulator (ONRSR) is reminding all operators of the importance of a system engineering approach.
With various subsystems - such as track, signalling, rolling stock, electrification, stations, depots, and control centres - closely interlinked, any change in one may affect the operation of another. As such, it is important to carefully consider the interfaces and how the subsystems interact with each other (including how these systems work together with people).
It is essential to understand the hazards when making system changes or introducing new products into a system and the effect such a change will have on the overall risk profile of the railway.
One particular area operators should pay attention to is the acceptance of products or systems based on cross-acceptance. That is, where a product or system is deemed safe because it has been applied safely on another railway or because it is compliant with appropriate standards.
Whilst cross acceptance can be an indication of performance, it cannot be taken as evidence that a product will perform safely in the particular railway system it is introduced to. As part of a robust engineering change process it is, therefore, important to understand the potential hazards a product or system may present in the environment it is introduced to - and the effects it might have on the overall safety risk of the railway.
Operators should demonstrate that they use appropriate systems engineering processes and safety assurance processes (e.g. EN50126/8/9 for complex systems) in their design and procurement approach. This can be achieved through the creation of a systems engineering management plan which specifies the procedures to identify and record stakeholders, system requirements and safety needs.
On 3 August 2020 ONRSR published two related fact sheets:
The Safety Critical Software Assurance fact sheet is designed to help rail transport operators ensure their safety management systems address the complexity of software systems along with its compliance and safety risk. It features a series of international lessons learned to illustrate key points.
The Systems Integration fact sheet focuses on the importance of a robust approach to systems integration in the context of major projects and other initiatives that are delivering complex and/or multifaceted safety systems. The aim of the resource being to ensure new technologies work together safely with existing railway infrastructure and rolling stock.
Additional safety action by the Rail Industry Safety and Standards Board
In January 2019, the Rail Industry Safety and Standards Board (RISSB) issued the Rolling Stock Safety Assessment Guideline as ‘an aid to rail industry describing common practice for the safety assessment of rolling stock and approvals.’ It set guidance for:
providing rolling stock safety assessment awareness in rolling stock lifecycle,
preparing and undertaking a safety assessment and safety assurance case toward regulatory compliance,
addressing stakeholder responsibilities for safety in the rolling stock lifecycle.
The Rolling Stock Safety Assessment Guideline listed several systems and safety engineering standards, including EN 50126-1, as normative references.
In June 2020, the RISSB-developed Australian Standard (AS) 7473 Complex system integration in railways was issued. The standard is freely available to RISSB member organisations. It stated:
The objective of this Standard is to establish an industry approach for managing:
a. the risks associated with integrating complex systems;
b. the design and implementation of complex system interfaces; and,
c. the planning, conducting and reporting on system integration testing (SIT).
This Standard defines an approach to support the preparation and execution of system integration for rail projects in Australia. It provides processes to support the definition, control and optimization of integration processes used within an organization or project that can be applied by the adopter when delivering railway systems.
This Standard is targeted at railway systems integrators such as operators, delivery authorities, prime contractors and alliances, or other bodies involved in integrating systems for or into a railway environment. Specifically, activities that result in changes or creation of railway configuration or operation.
AS 7473 listed two systems and safety standards as normative references: British Standard (BS) EN 50126 and ISO/IEC 15288.
BHP Train Control Incident and Emergency Response Procedure, 0090625, version 8, August 2018.
BHP Electronically Controlled Pneumatic Braking (ECPB) Project, Definition Phase Study Report, Version 1.0, May 2014.
BHP Electronically controlled pneumatic braking (ECPB) project, Selection phase to definition phase IPR version 1.2, 6 December 2013.
BHP - Testing of ECP brakes for BHPB-IO, Report 2012/657, April 2012.
Minerals Australia, Western Australia Iron Ore, Risk Management Procedure, Document 0126027, Version 4.0, February 2018.
New York Air Brake, Maintenance and Repair, EP-60 Freight Car Integrated Brake Control System, IP – 234 Rev (3/17/17) – en.
Vlasblom JID, Pennings HJM, Van der Pal J and Oprins EAPB (2020) ‘Competence retention in safety-critical professions: A systematic literature review’, Educational Research Review, 30:10.1016.
Watson NF, Badr MS, Belenky G, Bliwise DL, Buxton OM, Buysse D, Dinges DF, Gangwisch J, Grandner MA, Kushida C, Malhotra RK, Martin JL, Patel SR, Quan SF, Tasali E (2015) ‘Joint consensus statement of the American Academy of Sleep Medicine and Sleep Research Society on the recommended amount of sleep for a healthy adult: methodology and discussion’, Journal of Clinical Sleep Medicine, 11:931-952.
Welschen R, Bellon E, Brown C, Fullalove R, Kennedy G, Irvine K, Mumford N, Nadeem M, Nasr J, Tildesley E, Patel H, Roodt D (2021) An overview of systems engineering in the Australian transport sector, Systems Engineering Society of Australia, version 5.0.
Wickens CD, Hollands JG, Banbury S and Parasuraman R (2013) Engineering psychology and human performance, 4th edition, Pearson Boston, MA.
Wisher RA, Sabol MA and Ellis JA (1999) Staying sharp: Retention of military knowledge and skills, US Army Research Institute, Special Report 39.
Submissions
Under section 26 of the Transport Safety Investigation Act 2003, the ATSB may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. That section allows a person receiving a draft report to make submissions to the ATSB about the draft report.
A draft of this report was provided to the following directly involved parties:
BHP Billiton Iron Ore Pty Ltd
Office of the National Rail Safety Regulator
the driver of M02712.
Submissions were received from:
BHP Billiton Iron Ore Pty Ltd
the Office of the National Rail Safety Regulator.
The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.
Appendices
Appendix A – Operating Instruction 18-72
Appendix B – Research associated with various roster patterns
Research associated with night shifts
Shift work is an inevitable part of commercial transport. However, night shifts will generally have a negative effect on a person’s amount of sleep, sleepiness and performance (Akerstedt and others 2009, Sallinen and Kecklund 2010). The primary reason is that people are generally adapted to a normal sleep-wake cycle (with sleep at night), and a night shift forces people to work and sleep at the physiologically least suitable times of day.
A range of factors can influence the severity of problems associated with night shifts. Although some concern has been expressed about the length of shifts (such as 12-hour shifts versus shorter shifts), research has shown that the length of a shift itself is not necessarily as problematic as other features of roster patterns, the nature of the work and the frequency of rest breaks (Ferguson and Dawson 2012). Tucker and Folkard (2012) stated:
Work schedules that conflict with the normal sleep-wake cycle can result in considerable cumulative fatigue that can only be dissipated if the timing of rest periods allows adequate sleep… although it is clearly possible to make recommendations for each specific feature, the impact of the features of any given work schedule really need to be considered in combination with one another. For example, a span of five successive 12-hour shifts might be perfectly acceptable if there are frequent rest breaks and they are worked during the day, but totally unacceptable if there are no rest breaks and they are worked at night.
One significant feature is the number of consecutive night shifts. Research has shown that most people will generally not adapt their sleep-wake cycle while on night shifts, and therefore, as the number of consecutive night shifts increases, the more problematic the effects associated with the reduced amount of sleep each day (Tucker and Folkard 2012). Ferguson and Dawson (2012) cited studies showing people on 12-hour night shifts get between 5 and 6.5 hours sleep.
Another significant feature is the timing of the recovery break between consecutive night shifts. As noted by Tucker and Folkard (2012):
…night-shift workers experience greater sleep problems when they go to bed in the relatively "late" morning, after returning home from the night shift... Sleep propensity falls rapidly from its peak between 4 and 6 a.m. until about 1 p.m., implying that the later nightworkers go to bed following a night shift, the more difficulty they may have in falling asleep. They may also find it difficult to stay asleep long enough to recover adequately.
Roach and others (2003) found that train drivers with a 12-hour break between 2 shifts had an average of 5.2 hours sleep, but the amount of sleep significantly varied depending on the start time of the break. Breaks beginning at 0800–1000 had the lowest amount of sleep (3.1 hours), and breaks starting from 0400–1400 had less than 5 hours sleep. Other research has also noted a similar relationship between the time of a recovery period and the amount of sleep obtained (Spencer and others 2006).
Shifts that begin in the early morning can also be problematic as people generally go to bed at (or cannot get to sleep until) their normal bedtime and they get less than their normal amount of sleep (Tucker and Folkard 2012). Research has shown that early morning shifts are associated with elevated levels of fatigue risk and higher self-ratings of fatigue compared to day shifts (Sallinen and Hublin 2015).
Some researchers have stated that the number of consecutive night shifts or early shifts be limited to a maximum of 3 in a row (Tucker and Folkard 2012). Others have recommended that rosters with several consecutive early morning starts be avoided where possible (Roach and others 2011).
Research associated with roll-over roster patterns
A significant amount of research has been conducted on the effects of roll-over roster patterns in FIFO work in the offshore oil and gas industry, primarily in the North Sea (Fossum and others 2013, Parkes 2012, Parkes 2015). For these environments:
Shifts generally started at about 1800–1900 (night shift) or 0600–0700 (day shift).
Roll-over patterns starting with night shifts typically resulted in an average of 6.5 hours sleep after each night shift.
Many workers started adapting their sleep-wake cycle to the night shift after 5–6 nights. The adaptation generally occurred in these environments because much of the work was indoors and there was limited exposure to sunlight. After rolling over to the day shifts the workers then had to re-adapt to day shifts, which led to further sleep restrictions.
Roll-over patterns starting with night shifts were found to lead to less sleep over the fortnight than roll-over patterns starting with day shifts or roster patterns with 14 night shifts. However, workers generally preferred roll-over patterns commencing with night shifts because they were adapted to a normal sleep-wake cycle when they went home.
A recent guidance document on FIFO work for the oil and gas industry (IPIECA 2015) discussed a number of risk factors. It recommended avoiding rotating from nights to days during the middle of a 14-day roster pattern. It also stated that ‘Long tours of night shifts are associated with cumulative fatigue due to a lack of restorative sleep.’ Recommended control measures included ‘adequate and regular breaks’ within each shift, and to avoid scheduling safety-critical work during 0200-0600.
There has been less research conducted into roll-over roster patterns in onshore environments. Ferguson and others (2010, 2012) examined FIFO roll-over roster patterns in a Western Australia mining environment with day shifts 0545–1800 followed by night shifts 1745–0600. The average amount of sleep following a day shift (6.1 hours) was less than following a night shift (5.7 hours).[45]The amount of sleep did not increase over the week of night shifts, and various measures indicated that the workers’ sleep-wake cycles did not adapt during the week of night shifts. Performance on reaction time tasks also decreased over the week of night shifts. The authors noted that the workers were exposed to a significant amount of sunlight after completing their night shifts and before getting to bed, whereas workers in the North Sea environment would have less exposure to sunlight.
Very little research has examined the influence of different start times for roll-over roster patterns. Paech and others (2014) examined the influence of start times for FIFO train drivers at a Western Australia mining site. All the swings started with morning shifts (that is, starting from 0100–1200) followed by afternoon shifts (starting from 1300 to 0000). Workers obtained an average of 6.1 hours sleep each break between shifts, with the amount of sleep varying depending on the start time of the break. Breaks beginning from 1000 to 1200 were associated with 4.4 hours sleep and breaks starting from 0100 to 0300 were associated with 6.8 hours sleep. Overall, breaks starting from 0400 to 1200 were associated with less than 6 hours sleep whereas breaks starting from 1300 to 0300 were associated with more than 6 hours sleep.
Summary
In summary, roll-over roster patterns that include 7 consecutive 12-hour night shifts present an elevated risk of fatigue, particularly in environments such as Australian mining sites where it is unlikely that workers will adapt their sleep-wake cycles during a week of night shifts due to exposure to sunlight. This risk is further exacerbated depending on the start and end time of the shifts, with night shifts ending in the late morning likely to lead to the least amount of sleep.
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
Preliminary report
Report release date: 12/03/2019
This preliminary report details factual information established in the investigation’s early evidence collection phase and has been prepared to provide timely information to the industry and public. Preliminary reports contain no analysis or findings, which will be detailed in the investigation’s final report. The information contained in this preliminary report is released in accordance with section 25 of the Transport Safety Investigation Act 2003.
The occurrence
Overview
At approximately 0440[1] on 5 November 2018, loaded BHP ore train M02712 rolled away from the 210.7 km mark located near Garden South on the Nelson Point to Newman railway, Western Australia. There was no driver on board the train at the time. The train travelled uncontrolled on the west track for about 91 km before Hedland train control decided to derail the train by routing it from the west track to the east track at a crossover located at Turner South.
At about 0526, the head end locomotives traversed the crossover. Shortly after, 245-ore cars and the two remote locomotives, located mid consist, derailed. There was significant damage to rolling stock and track infrastructure at Turner South (Figure 1). There was no injury to any person from the derailment.
Figure 1: Train M02712 wreckage at Turner South
Aerial image viewed in a southerly direction of train wreckage and track damage at Turner South. The lead locomotives 4420 and 4434 and one ore car (out of frame in the foreground) remained on track.
Source: BHP, annotated ATSB
Sequence of events
At about 0337 on 5 November 2018, train M02712 was travelling at 60 km/h on the west track approaching the BHP access road level crossing at the 211.6 km mark. The driver had set the throttle control for maximum dynamic braking and commenced moving the Electrically Controlled Pneumatic (ECP) braking control toward a 39 per cent application.
At about 0339, communication between the lead locomotive and the combined end of train monitor (CEOT) was lost, triggering an automated 120 per cent ECP emergency brake application, stopping the train as it approached Garden South. Shortly after, the driver made an emergency radio call to Hedland train control reporting the occurrence, his location (at the 210.737 km mark between Shaw and Garden), and the details of alert messages displayed to him by the locomotive on-board systems.
The train controller placed blocks to signals on the adjacent east track between Garden South and Shaw North to protect the train (from other rail movements) and contacted personnel from the Redmont[2] maintenance gang to assist the driver. The controller advised the driver that assistance was en route and requested he confirm the train’s location from a kilometre mark[3] closest to the lead locomotive.
The driver stated that the FIRE[4] system displayed 210 km, but would detrain and check the kilometre mark on the ground to confirm. At about 0351, the driver placed the reverser[5] control to the centre (neutral) position, turned the generator field[6] off and fully applied the locomotive independent brake before exiting the locomotive cab. The 120 per cent emergency brake application was active and the automatic brake handle remained set at the position equating to a 39 per cent ECP brake application.[7]
After receiving confirmation of the 210.7 km mark, the controller instructed that 101 per cent handbrakes[8] were required to secure a loaded train on the falling track grade. The controller asked the driver if he wanted to start applying them now or go back up to the locomotive and wait for the arrival of personnel from the Redmont gang. At about 0353, the driver decided to commence applying handbrakes to the 268 ore cars from the front of the train.
At about 0355, an empty ore train (M02727) travelling on the adjacent east track toward Yandi Junction stopped at Garden South due to the blocking protections set up previously. About 30 minutes later, personnel from the Redmont gang advised train control of their arrival at the 210 km mark to assist the driver in applying handbrakes. The train controller suggested the gang start applying handbrakes from the rear of the train and proceed toward the driver who was working from the front.
Hedland control continued to maintain contact with the driver of M02712 at 10-minute intervals during which the driver advised that he had found a disconnection in the train-line cable[9]. The train-line cable was located on the opposite side of the train and not accessible safely, so the driver continued to apply handbrakes to secure the train. During one of the scheduled calls, the driver reported to the controller that the application of handbrakes was progressing well despite having trouble walking along the ballast shoulder next to the stationary train. The driver also reported that he was aware the Redmont gang had arrived to check the integrity of the rear of the train and to apply handbrakes. The driver said that he planned to continue working toward the locomotives mid train, report to train control then return to reinstate the break in the train-line cable.
At about 0440, the driver heard air venting from the ore car brakes and shortly after noticed the train begin to move forward. The driver first attempted a radio call to the Redmond gang alerting that the brakes had ‘bled off’ but there was no response. Shortly after train M02712 began to roll away, the ATP system requested a penalty brake application but it was ineffective in stopping the train.
About four minutes later, the driver of the empty ore train standing at Garden South (M02727) contacted train control advising train M02712 was moving and had passed his location at an estimated speed of about 50 km/h with brakes dragging.[10]
At 0446, train control received an emergency call from the driver of M02712 alerting that the brakes had bled off and the train was now a ‘runaway’. Train control acknowledged the emergency call and advised he had set signal GNN4 at Garden North to red, attempting to stop the train by triggering the locomotive on-board automatic train protection system. Train M02712 passed signal GNN4 at about 80 km/h and continued to increase in speed. Although the ATP system requested a penalty brake application in response to signal GNN4 at red and to an over speed, these penalty applications were also ineffective in stopping the train.
About 80 km ahead, another train (M02728) travelling on the eastern track was approaching Abydos North. Hedland control contacted its driver instructing him to stop, detrain and move to a safe place. Hedland control also contacted the drivers of the two other trains (M02729, M02710) operating between Garden North and Port Hedland, instructing the drivers to stop, detrain and move to a safe place. Trains M02729 and M02710 stopped at locations north of Turner (Figure 2).
At about 0502, the driver of the empty ore train (M02727) stopped at Garden South, contacted Hedland control advising that the Redmont gang had mistakenly applied handbrakes to his train rather than to train M02712.
Train M02712 continued through Spring and Coonarie reaching a speed of 162 km/h before slowing on the rising grades toward Woodstock (Figure 2). At about 0509, train M02712, travelling at about 128 km/h, passed over the level crossing at the 154.3 km mark before Woodstock South. After Woodstock, the track grade again began to fall toward Port Hedland and train M02712 gained speed to about 130 km/h as it passed train M02728 stopped at the 130.5 km mark on the eastern track north of Abydos.
At about 0520, Hedland control set the crossovers at Turner South and Turner North to switch train M02712 between adjacent tracks in an attempt to derail it as the traversed the crossover at speed. About six minutes later, the head end locomotives travelling at 144 km/h traversed the crossover at the 119.4 km mark at Turner South. Locomotives 4420, 4434 and the first ore car remained coupled and on track, travelling about 1.6 km further before stopping. Ore cars in position two to 134 of the first rake, the remote locomotives 4472 and 4440 and ore cars one to 112 from the second unit rake derailed near the crossover. The last 22-ore cars of the second unit rake remained coupled and on track.
The derailment destroyed two locomotives, 245-ore cars and 2 km of track infrastructure at Turner South.
Figure 2: Location map BHP Port Hedland railway
Source: BHP, annotated ATSB
Context
Train information
Train M02712
The ore train operated as a unit train weighing approximately 42,500 t and was 2,860 m long. It consisted of two SD70ACe type locomotives (4420, 4434) leading , a unit rake of 134-ore cars, two remotely operated SD70ACe type locomotives (4472, 4440) located mid-train, and a second unit rake of 134-ore cars. The ore train was operating between the loading facility at Mining Area C situated on the spur line extension from Yandi, and the unloading facility at Nelson Point, Port Hedland (Figure 2).
Locomotive on-board automatic train protection system
The four locomotives on train M02712 were each equipped with an Alstom Ultra-Cab II (UCII) microprocessor controlled automatic train protection (ATP) system. The UCII system was not a standalone system; it interfaced electronically with other on-board equipment including FIRE, Electrically Controlled Pneumatic (ECP) braking systems, wayside transponders[11] and other control systems that combined to provide for the safe operation of the train within the parameters defined in the BHP iron ore rules and regulations.
The ATP functions included monitoring the locomotive speed and supervising its operation within the limits imposed for the track section. If the locomotive was moving faster than the target speed limit, alarms would sound prompting the driver to reduce speed.
The locomotives carry a radio transmitter, transponder reader and antenna. Transmissions are relayed between the locomotive and transponders fastened to the track cross-ties (sleepers) at key locations such as ATP entry and exit points and interlocked wayside signals along the railway. Unique location identification and target speed data is relayed from the track mounted transponders to the locomotive UCII microprocessor.
The driver must reduce speed to the target limit within a predetermined time. If this does not occur, the ATP automatically interfaces with the braking system to initiate a brake application to stop the train. The type of brake application is dependent on the setup of the locomotive at the time of the command. If the locomotive is configured for conventional pneumatic braking, the ATP initiates a service brake application. If this was ineffective in slowing the train, the ATP then initiates a penalty brake application. For an ECP braking configuration, the ATP requests a penalty brake application.
Additionally, when the locomotive was stationary with its reverser in the neutral (centre) position and the ATP detected a train movement of more than 0.5 m, the ATP requested a penalty brake application to prevent a potential locomotive runaway.
Each ATP automatically configures to mirror the ECP brake setup for that locomotive as a head end unit (HEU), trail or remote unit. The ATP would not enforce target speed limits or runaway protection on locomotives configured as either a trail or remote unit. The ATP in each of the four locomotives in train M02712 (4420, 4434, 4472 and 4440) functioned respectively as a HEU, trail and two remote units.
Braking and Distributed Power systems
Train M02712 was equipped with an EP-60 New York Air Brake electrically controlled pneumatic (ECP) braking system. The system consisted of locomotive equipment, ore car braking control equipment, an end of train monitor, and a power and communications distribution system.
Locomotive equipment comprised a train-line communications controller, power supply and identification module. The lead locomotive 4420 functioned as the HEU. The HEU communicated with each of the 268-ore car braking control devices (CCD) and remote locomotives via embedded transmissions in the train-line cable comprised of a single set of wires forming the intra-train power and communications network. The CCD unit used 230 Volt Direct Current power from the train-line to charge its batteries and supply power to its electronics.
In ECP mode, the EP-60 system used the position of the HEU automatic brake handle to control the operation of the locomotive and ore-car brake cylinders. The FIRE system provided the interface to the driver displaying braking parameters related to the ECP system mode, alarms, diagnostic messages and brake command input.
The FIRE system displayed the level of brake command input as a percentage (%TBC)[12], typically a number between 0 and 100 per cent or as 120 per cent:
0% = Release
10% = Minimum Service
100% = Full Service/Penalty application
120% = Emergency.
The Combined ECP end of train monitor (CEOT) installed on the last ore car coupler marked the end of the train, provided a termination point for the train-line and a transducer for end of train information, such as brake pipe pressure, back to the HEU to establish the integrity of the train-line and train consist. The CEOT used 230 Volt Direct Current power from the train-line to charge its batteries and supply power to its electronics.
If the power from the train-line is lost, the CCD and CEOT devices each continue to operate on battery power until the batteries run low or a 60 minute time period elapses. The CCD and CEOT devices will then enter shutdown mode. When a CCD shuts down it releases its ECP brake application and relinquishes control of brake cylinder pressure to the conventional pneumatic braking system. If the brake pipe is charged and a pneumatic application is not in effect, the brake cylinder pressure will release.
The BHP locomotive fleet was equipped to enable control of multiple distributed power units within the train. Communication of synchronous control and indication signals between the HEU, trailing and remote locomotives, located mid consist, also occurred via the train-line system.
As a contingency, the ECP overlay braking system and train-line could be shut down and the HEU configured to communicate power and brake commands via UHF radio communications to the remote locomotives. This configuration disables ECP braking and train braking reverts to conventional pneumatic operation via the train brake pipe. The HEU configuration establishes communication with the CEOT by radio.
The locomotive independent brake handle is located immediately below the automatic brake handle and controls the locomotive braking independently of the automatic train brake. It also applies the brakes on other locomotives in the train. The independent brake control only apples to the locomotives (lead and remote) and not the ore cars in the train. The independent brake control operates pneumatically irrespective of the HEU configuration.
The independent brake control handle can be positioned to:
REL (release), releasing the locomotive brakes provided the automatic brake handle is also in the REL position
SERVICE, moving the handle through the service zone increases locomotive braking effort
APPL position applies full braking effort on the locomotive(s)
Bail off function, depressing the handle in either the REL position or SERVICE zone suppresses any automatic train brake application in progress on the locomotive(s).
Vigilance control
The vigilance control system functions to monitor driver activity and stop the locomotive/train when there was no response from the driver to aural and visual warnings displayed via the FIRE system. The vigilance system uses random timing and task linking to monitor driver activity.
The vigilance system is active when the locomotive air brake is set as HEU and the air brake cylinder pressure is less than a predetermined level. Vigilance system suppression occurs when the locomotive air brake cylinder pressure is greater than a predetermined level, when the braking system is set to trail or remote, when the reverser is in the centred position, or when the locomotive configuration is set for operation at a defined slow speed.
Track
The BHP iron ore railway is a standard gauge track structure constructed with continuously welded 68 kg/m rail fastened with resilient clips to concrete sleepers. The sleepers are contained in crushed rock ballast. The track structure configuration enabled the operation of rolling stock with 40 t axle load.
In the direction of travel, the track gradient from Mining Area C was primarily a rising grade approaching Shaw located in the Chichester Range before transitioning to a mainly falling grade toward Nelson Point. The roll away occurred between Shaw North and Garden South where the track gradient was -1.5 per cent—the steepest track gradient of the track section between Yandi Junction and Nelson Point.
The ATP governed the maximum permissible track speed for the various sections dependent on the mode of operation, with the target speed displayed to the driver via the FIRE system. The maximum track speed for the Newman to Port Hedland railway was 75 km/h for loaded ore trains.
Train control
BHP manages train movements remotely from its train control centre located in the Integrated Remote Operations Centre in Perth. The train control centre has five operational control areas (desks) identified as Hedland, Newman, 6PG, Hub control and Yard control. All communications between the Perth control centre, train movements, control systems and wayside equipment is via a dedicated VHF radio system.
The runaway occurred within the operational area managed by Hedland train control that extended from the 67 km mark located south of Walla to the 260 km mark located south of Cowra.
Safety action
The BHP preliminary investigation recommended a series of actions for implementation over the short (prior to recommencing movements of trains), medium (within approximately one year) and long term (over one year).
Short term
Issue communications to all train drivers regarding the release of ECP brakes under certain conditions.
Amend operating instruction related to Brake Pipe Emergencies and Penalties.
Medium term
Investigate improvements to ECP braking system Handbook in relation to any of the short term controls.
Review the processes for issuing operating instructions and disseminating information to drivers and assess value in presenting a measure of criticality with each operating instruction to avoid dilution of critical instructions amongst non-critical information.
Undertake a review of the ECP braking system timeout function to explore software options to increase timeout period at a sacrifice to ECP braking system battery performance.
Investigate a hardware/software solution whereby the on-board control system, FIRE or brake computer automatically dumps the air from the train brake pipe under different conditions.
Long term
In conjunction with the BHP existing signalling upgrade project, introduce new automatic train protection system on-board every locomotive, track maintenance machine and Hi-rail in the BHP fleet.
Ongoing investigation
The ATSB investigation has obtained relevant material and conducted interviews with a number of BHP staff. The ATSB is continuing to gather documentation about the design of train braking systems operated by BHP and the procedures contained in the BHP safety management system for the management of risk and the development and dissemination of safety critical information to rail safety workers.
The ongoing ATSB investigation will include consideration of the following:
examination of factors associated with train-line cable connectors
design, operation and serviceability of the locomotive and ore car electrically controlled pneumatic and pneumatic braking systems, and the interface between them at time of the occurrence
effectiveness of critical control management process in identifying and managing operational risk from a runaway occurrence
arrangements for the dissemination of safety critical information and associated driver training
effectiveness of recovery controls—runaway protection
effectiveness of emergency management plan response systems and actions
factors influencing the driver’s response to a penalty brake application
train handling, driver qualifications, experience and health information
Status of BHP short, medium and long term actions following the occurrence.
_____________
The information contained in this interim report is released in accordance with section 25 of the Transport Safety Investigation Act 2003 and is derived from the initial investigation of the occurrence. Readers are cautioned that new evidence will become available as the investigation progresses that will enhance the ATSB's understanding of the accident as outlined in this report. As such, no analysis or findings are included.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
On 31 October 2018, an Aeropro 3K, registration 24-7502, collided with terrain 65 km north of Wentworth in NSW. The pilot and passenger were fatally injured.
Recreational Aviation Australia (RAAus) commenced an investigation of this accident and requested technical assistance from the Australian Transport Safety Bureau (ATSB) to download the flight data from a Dynon data logging unit.
To protect the information supplied by RAAus to the ATSB and the ATSB's investigative work to assist RAAus, the ATSB has initiated an investigation under the Transport Safety Investigation Act 2003.
Any enquiries relating to the accident investigations should be directed to RAAus at: www.raa.asn.au.
Updated: 26 March 2019
The ATSB has completed its work attempting to download the recorded data from the Dynon SkyView SV‑D1000 unit supplied by RAAus. A report detailing the work undertaken by the ATSB was provided to RAAus on 15 March 2019.
On 11 October 2018, a Boeing 787-9, 9V-OJE, operated by Scoot Tigerair (Scoot), departed Singapore on a scheduled flight to Perth, Western Australia. During descent, the flight crew noticed that the right engine was slow to respond to commands, and its performance continued to decline throughout the descent. While passing through 9,000 ft, severe thrust asymmetry developed, and the engine shut down shortly afterwards. The crew followed appropriate procedures, and due to the proximity of the airport, elected not to attempt a restart. The aircraft landed safely with emergency services in attendance. There were no injuries sustained and no aircraft damage as a result of the incident.
What the ATSB found
The ATSB determined that following a series of engine status and alert messages, 9V-OJE experienced an uncommanded engine shutdown while on descent into Perth, before landing safely using the operational engine.
Based on a review of the flight data and an examination of engine components by Rolls-Royce, the engine shutdown was due to debris from worn journal bearings in the engine’s secondary high-pressure fuel pump blocking an inlet filter for the fuel metering valve servo assembly. This prevented the valve from delivering sufficient fuel to the engine.
Rolls-Royce also determined that, between late 2018 and early 2019, the operator’s fleet of 787 aircraft had been particularly susceptible to low‑life wear in the journal bearings of the secondary high‑pressure fuel pump. It identified a number of potential factors that led to the component wear but, due to the number of variables, a single/dominant reason could not be established.
What has been done as a result
Rolls-Royce updated its Fault Isolation Manual to instruct all operators to remove the fuel pump and hydro-mechanical unit in the event of a maintenance message regarding the fuel metering valve not being in the commanded position. Rolls-Royce is also monitoring maintenance messages and investigating the possibility of using flight data to detect fuel pump journal wear before its effects on valve operation become apparent.
Safety message
This occurrence highlights the importance of flight crew being familiar with emergency procedures, so that the appropriate corrective action can be taken quickly and effectively. In this case, the flight crew worked effectively to assess the situation, and took appropriate action to minimise risk in accordance with the operator’s flight crew operations manual.
This occurrence also shows that positively identifying the factors contributing to technical failures can be difficult and time consuming. However, manufacturers and operators can implement interim risk mitigation measures, as was the case here.
The investigation
Decisions regarding whether to conduct an investigation, and the scope of an investigation, are based on many factors, including the level of safety benefit likely to be obtained from an investigation. For this occurrence, a limited-scope investigation was conducted in order to produce a short investigation report, and allow for greater industry awareness of findings that affect safety and potential learning opportunities.
The occurrence
On 11 October 2018, at about 1421 Western Standard Time,[1] a Boeing 787-9, 9V-OJE, operated by Scoot Tigerair (Scoot), departed Changi Airport, Singapore. The aircraft was on a scheduled passenger flight to Perth, Western Australia, with 11 crew members and 356 passengers on board.
Approximately 2 hours into the flight, the crew received two status messages indicating abnormalities within the right engine. Three hours later, during descent, the aircraft was passing through FL 250[2] when the crew noticed that the right engine was slow to respond to commanded inputs. Throughout the descent, the right engine performance continued to decline. Passing through 9,000 ft, severe thrust asymmetry developed, and the captain noticed rudder input from the autopilot. Shortly after, at 1853, the crew received the engine‑indicating and crew‑alerting system (EICAS) message ENG FAIL R, and the right engine shut down.
In response, the flight crew declared a PAN[3] and requested air traffic control clearance to level off at 5,000 feet and be vectored off the approach to allow time for completion of the quick reference handbook (QRH) checklist items. Completion of the QRH checklist required the flight crew to decide whether they should attempt to relight the engine. Due to the proximity of the airport and because the aircraft is capable of landing safely with one engine, the flight crew decided that attempting an engine restart was unnecessary. After the checklist was completed, the flight crew conducted a NITS[4] briefing with the cabin crew.
Subsequently, the flight crew completed the landing performance calculations and advised ATC that they were ready to land. The flight crew also requested that emergency services conduct a visual inspection of the aircraft after landing.
At 1909, the aircraft landed safely at Perth Airport and emergency services carried out a visual inspection. The aircraft was cleared to taxi to the parking bay and disembark passengers normally via the aerobridge.
There were no injuries sustained and no damage to the aircraft as a result of the occurrence.
Context
Subsequent maintenance
Following the occurrence, an engineering team carried out a detailed inspection of the aircraft to address the in‑flight shutdown and status/EICAS warning messages observed by the crew. The technical examination resulted in replacement of the right engine hydro‑mechanical unit (HMU) and a high‑power engine run was then successfully performed.
On 12 October 2018, a non-revenue flight (no passengers or cargo) was conducted to return the aircraft to Singapore for further maintenance, during which time the electronic engine controller (EEC) was replaced. The aircraft then returned to revenue service.
On 15 October 2018, the aircraft was on a flight from Sydney to Singapore when several maintenance messages indicating similar issues to the occurrence flight were generated, but there was no noticeable effect on engine performance. Following the flight, additional components were replaced, including the:
HMU (further replacement)
fuel pump
high‑ and low‑pressure fuel filters
left and right variable stator vane actuator.
The aircraft was then declared serviceable and returned to service with no further recurrence of the maintenance messages.
Engine fuel system
The Trent 1000 fuel system includes a three-stage pump that supplies fuel from the aircraft to the engine. Fuel runs through a low pressure (LP) pump followed by two high pressure (HP) pumps, identified as primary and secondary, running in parallel. The primary HP pump operates under all conditions, while the larger secondary HP pump increases fuel flow to the engine at periods of high demand, such as take-off.
The HP pumps supply fuel to the HMU, which controls fuel flow to the engine using its fuel metering valve (FMV) as follows:
Fuel enters the FMV servo assembly within the HMU through an inlet filter.
To change the flow rate of fuel supplied to the engine, the EEC sends electrical signals to the FMV servo assembly.
The signals control the position of a valve, which changes the fuel pressures within the servo assembly.
These servo pressures determine the position of the FMV, which ultimately controls the flow rate of fuel to the burners.
Rolls-Royce investigation
Following the occurrence and subsequent non-revenue flight to Singapore, the engine manufacturer, Rolls-Royce, conducted an investigation into the occurrence. This included reviewing flight data from both flights, examining engine components from 9V-OJE, and based on its findings, assessing the Trent 1000 fleet more widely.
Review of flight data
Approximately 2 hours into the flight, during cruise, the right engine’s EEC generated the following message:
Hydro-Mechanical Unit (Right Engine) fuel metering valve (FMV) torque motor current is too low or too high.
This message indicated that the current required to adjust the fuel flow via the FMV was outside the expected range. Eleven minutes later, another message indicated that the current had exceeded an allowable limit:
Hydro-Mechanical Unit (Right Engine) fuel metering valve (FMV) torque motor current is failed too low or too high.
During descent, approximately 3 hours later, two more messages were generated:
Hydro-Mechanical Unit (Right Engine) fuel metering valve (FMV) is not in commanded position.
Right Engine is failed below idle with fuel switch on.
Rolls-Royce determined that the first message was evidence the FMV was taking longer than it should have to reach the position specified by the EEC. It was found that the second message was generated after the EEC had commanded a deceleration. The FMV moved below the idle position as requested, but once deceleration had occurred, it did not move back to the directed idle position. The right engine then ran at sub-idle speed for a short time before shutting down. Data for the entire occurrence flight indicated that the torque motor current required to control the FMV position increased throughout the flight up until the in‑flight shutdown.
Rolls-Royce also identified that the maintenance messages generated during the flight to Singapore on 15 October 2018 indicated that control of the new FMV was still requiring a higher-than-expected torque motor current. However, the engine continued to operate normally, and the flight was completed without incident.
Component examination
Rolls-Royce examined the HMU from the occurrence flight (HMU 1) as well as the one from the subsequent flight (HMU 2). In both units, a build-up of metallic debris was found in various locations, although more debris was found in HMU 1. The inlet filter to the FMV servo assembly was at least partially blocked with debris in both units.
Rolls-Royce concluded that the in-flight shutdown of 9V-OJE’s right engine was the result of the blocked inlet filter on the FMV servo assembly. The blockage restricted the EEC’s ability to control the FMV, and ultimately, the flow of fuel to the engine.
The FMV servo assemblies from each HMU were scanned using CT imaging. The resulting x-ray cross sections are shown with photographs of each servo assembly in Figure 1.
Figure 1: Blocked inlet filters on both FMV servo assemblies
The build-up of metallic debris was greater on HMU #1. Some deformation can also be observed in the #1 inlet filter x-ray image. Rolls-Royce determined that this was likely due to the high‑pressure differential caused by the blockage.
Source: Rolls-Royce
Analysis of the metallic debris revealed that it consisted of material from the fuel pump bearings and the casing. All three stages of the pump (the LP pump, and the two HP pumps) were disassembled and examined by Rolls-Royce in the presence of the United Kingdom Air Accidents Investigation Branch.
The examination found that the debris originated from the secondary HP pump. The bearings for the secondary HP pump driven gear were heavily worn, with evidence of scoring and missing material (Figure 2). Rolls-Royce found that the casing for the driven gear had more wear than would be expected during normal operation, likely due to shaft movement resulting from the damaged journal bearings. No damage was found on other bearings within the secondary HP pump or the other two fuel pumps.
Figure 2: Worn journal bearings and casing for the secondary HP pump driven gear
Source: Rolls-Royce
Rolls-Royce reviewed the manufacturing records for the secondary HP pump but found that it was typical of the fleet. No abnormalities had been noted, and the pump dimensions were within the accepted tolerances. The material composition of the fuel pump components was checked and found to be similar to the rest of the population. The fuel pump and HMU from the left engine were removed and inspected as a precaution, but there was no evidence of journal wear or debris build-up.
Trent 1000 fleet inspection
On 1 November 2018, another Scoot Boeing 787 generated maintenance messages related to the HMU during start-up, prior to a flight. The engine was inspected, and some wear was also found on the secondary HP pump journal bearings.
To search for similar HMU maintenance messages, Rolls-Royce examined all maintenance data across the fleet of Trent 1000 Package B and Package C engines and continued to monitor ongoing flights. Six other events were found where messages were generated due to fuel pump debris blocking the FMV servo assembly inlet filter. Five of these events were from aircraft operated by Scoot, while one was from a different operator.
Of the events found in the Scoot fleet, the age of the pumps varied between 5,201 and 12,686 hours. The recommended life of the pumps was 22,000 hours. Based on the number of occurrences compared with the greater Trent 1000 fleet, Rolls-Royce determined that the secondary HP pump journal bearings on Scoot aircraft were particularly susceptible to low life journal wear.
In an effort to determine what was increasing wear susceptibility in the Scoot fleet’s bearings, Rolls‑Royce identified a number of potential factors, including the following:
Pump manufacture and build: The worn pumps found on Scoot aircraft had been manufactured over a number of years from 2015 to 2017. As such, it was determined that a batch or build issue was unlikely to be a common factor.
Fuel quality: Analysis of fuel samples from Singapore Changi Airport found no anomalies within the 12 months prior to the occurrence involving 9V-OJE. There were also no reports of fuel pump bearing wear from other Trent 1000 operators that used the same airport.
Operations: Rolls-Royce noted that Scoot generally flew shorter routes than most other Trent 1000 operators, but there were comparable operations with no evidence of fuel pump journal wear. Within the Scoot fleet, aircraft flew to multiple destinations, and there were no specific city pairs associated with the engines with worn bearings.
Maintenance: Scoot shared its maintenance facilities with another operator that also used Trent 1000 engines. There was no evidence of fuel pump bearing wear from this operator.
Based on its investigation, Rolls-Royce concluded the following:
It is likely that a combination of factors have led to Scoot bearings being particularly susceptible to significant low life wear, but analysis of data to date has not identified any significant differences between worn and unworn bearings, both within the Scoot fleet and the wider Trent 1000 Pack B & C fleets.
It was further noted that the majority of Scoot events occurred between late 2018 and early 2019. With the exception of the occurrence flight, none resulted in an in-flight shutdown.
The Rolls-Royce investigation also considered factors in addition to those listed above but, due to the number of variables, was unable to identify which might have been dominant with respect to the pump bearing wear. However, it identified and implemented interim measures (mainly related to engine data monitoring) to address the risk from low life wear of bearings.
Safety analysis
While on descent into Perth, the right engine of 9V-OJE shut down. After completing the necessary checklists, the flight crew landed the aircraft safely on one engine. There were no injuries sustained as a result.
The engine manufacturer, Rolls-Royce, concluded that the engine shutdown was the result of a blocked inlet filter on the fuel metering valve (FMV) servo assembly. This blockage restricted the electronic engine controller’s (EEC) ability to adjust fuel pressures within the servo. As a result, the EEC had limited control over the FMV position, and consequently the amount of fuel flowing to the burners. When the EEC commanded the FMV to increase fuel flow from sub‑idle to idle levels, it did not respond in time, and the engine shut down. The blockage was due to debris from worn journal bearings in the secondary HP fuel pump driven gear.
Rolls-Royce’s examination of flight data and maintenance records from its Trent 1000 engines identified that Scoot’s fleet of 787 aircraft had been particularly susceptible to low life wear in their secondary HP pump journal bearings over a period of several months. The Rolls-Royce investigation identified various potential factors that might have contributed to low life journal wear, including the fleet’s operation, maintenance, fuel quality, or pump design and construction. However, it found no evidence that any factors were significantly different to the wider Trent 1000 fleet. Additionally, due to the number of variables associated with operations, maintenance, design and manufacture, it was not possible to determine the relative effect of these factors (and possibly others) when combined.
Findings
ATSB investigation report findings focus on safety factors (that is, events and conditions that increase risk). Safety factors include ‘contributing factors’ and ‘other factors that increased risk’ (that is, factors that did not meet the definition of a contributing factor for this occurrence but were still considered important to include in the report for the purpose of increasing awareness and enhancing safety). In addition, ‘other findings’ may be included to provide important information about topics other than safety factors.
These findings should not be read as apportioning blame or liability to any particular organisation or individual.
From the evidence available, the following findings are made with respect to the uncommanded engine shutdown involving Boeing 787-9, 9V-OJE, on 11 October 2018 near Perth Airport.
Contributing factors
Following a series of status and alert messages related to the aircraft’s right engine, the engine shut down during descent. The flight crew followed the appropriate procedures and landed the aircraft safely using the operational engine.
The engine shutdown was the result of insufficient fuel delivery due to low pressure in the fuel metering valve servo assembly, as debris from worn fuel pump bearings had blocked its inlet filter.
The engine manufacturer, Rolls-Royce, identified that between late 2018 and early 2019 the operator’s fleet of 787 aircraft were particularly susceptible to low life wear in the journal bearings of the secondary high‑pressure fuel pump.
Other finding
Rolls‑Royce identified a number of potential factors that led to the component wear but, due to the number of variables, a single/dominant reason could not be established.
Safety actions
Whether or not the ATSB identifies safety issues in the course of an investigation, relevant organisations may proactively initiate safety action in order to reduce their safety risk. The ATSB has been advised of the following proactive safety action in response to this occurrence.
Safety action by Rolls-Royce
As part of its investigation into the engine failure, Rolls-Royce instructed Scoot to remove the engine’s fuel pump in the event of debris being found in the ports during removal of a hydro‑mechanical unit. In February 2020, the Fault Isolation Manual was updated to instruct all operators to remove the fuel pump and hydro-mechanical unit in the event of a maintenance message regarding the fuel metering valve not being in the commanded position.
Rolls-Royce is investigating the possibility of detecting potential fuel pump bearing journal wear by using flight data (particularly the fuel metering valve torque motor current) to detect partial filter blockage before maintenance messages are generated. It is also continuing to monitor maintenance messages and the condition of unserviceable fuel pumps ‘to ensure that the risk of an in-flight shutdown caused by fuel pump bearing wear is maintained at an acceptable rate’.
Sources and submissions
Sources of information
The sources of information during the investigation included:
the aircraft captain
Scoot Tigerair
Rolls-Royce.
Submissions
Under section 26 of the Transport Safety Investigation Act 2003, the ATSB may provide a draft report, on a confidential basis, to any person whom the ATSB considers appropriate. That section allows a person receiving a draft report to make submissions to the ATSB about the draft report.
A draft of this report was provided to the following directly involved parties:
the aircraft captain
the aircraft first officer
Scoot Tigerair
Rolls-Royce
The Boeing Company
the Civil Aviation Safety Authority
the Transport Safety Investigation Bureau of Singapore
the United States National Transportation Safety Board
the Air Accidents Investigation Branch (United Kingdom).
Submissions were received from:
Rolls-Royce
the United States National Transportation Safety Board
the Air Accidents Investigation Branch (United Kingdom).
The submissions were reviewed and, where considered appropriate, the text of the report was amended accordingly.
Purpose of safety investigations
The objective of a safety investigation is to enhance transport safety. This is done through:
identifying safety issues and facilitating safety action to address those issues
providing information about occurrences and their associated safety factors to facilitate learning within the transport industry.
It is not a function of the ATSB to apportion blame or provide a means for determining liability. At the same time, an investigation report must include factual material of sufficient weight to support the analysis and findings. At all times the ATSB endeavours to balance the use of material that could imply adverse comment with the need to properly explain what happened, and why, in a fair and unbiased manner. The ATSB does not investigate for the purpose of taking administrative, regulatory or criminal action.
Terminology
An explanation of terminology used in ATSB investigation reports is available here. This includes terms such as occurrence, contributing factor, other factor that increased risk, and safety issue.
Publishing information
Released in accordance with section 25 of the Transport Safety Investigation Act 2003
Ownership of intellectual property rights in this publication
Unless otherwise noted, copyright (and any other intellectual property rights, if any) in this report publication is owned by the Commonwealth of Australia.
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright, this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form licence agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you wish to use their material, you will need to contact them directly.
