Engineering verification of railcar system serviceability before return-to-service

Maintenance contractor response

Thank you for your contact about Critical System Verification processes. 

[The maintenance contractor provided] the process for Railcar Fault Management ([document 1]) detailing our standard fault management process for [type 2] railcars. (Supplied to the ATSB and ONRSR)

We have also summarised below some examples of Train System Verification and specifically as mentioned the Automatic Train Protection system (ATP).

The [type 2] railcar has several self-check safety functions, ATP extract below, we have also attached (supplied to the ATSB and ONRSR) a copy of the maintenance instructions ([document 2] Section [x] and [xx] relate to system safety functionality). 

ATP Failure 

ATP system problems can be generally grouped into the following categories:

1. Failing to successfully complete the ATP system start-up tests when the cab is activated.

2. Problems indicated by the ATP system in service (operational, speed input or transponder problems).

3. Vehicle fault indication system indicating an 'ATP recorder fault'

Because the ATP system is designed as a failsafe system, serious ATP system faults will cause brakes to be applied to the vehicle. To allow the vehicle to be driven when there is a serious ATP system fault, there is a lead sealed ATP ISOLATE switch in each driving cab to isolate the system. Operation of the ATP isolate switches is recorded by the vehicle computer in the fault memory.

Cab Activation Startup Tests

The ATP system includes a built-in test program that is performed every time a cab is activated. The start-up tests ensure that the ATP system is functioning correctly before the vehicle is allowed to move. 

The start-up test sequence is divided into nine stages. The first seven test stages (0 to 6) are indicated by displaying the corresponding number (0 to 6) in the middle character position of the main display on the driver control panel. All nine stages (0 to 8) are indicated by displaying the corresponding number on the comparator cassette display.

The main display panel cannot normally display the characters 7 and 8 during start-up test sequence stages 7 and 8 respectively, because the display must remain blanked to simulate a system failure.

The [type 2] railcar has several self-check safety functions, across multiple critical systems, we have also attached a copy of the Safety and Monitoring Technical Information ([document 3]) detailing several systems. (Supplied to the ATSB and ONRSR)

[The maintenance contractor] also provided a copy of the Rollingstock Management Plan. (Supplied to the ATSB and ONRSR) The Rollingstock Management Plan (RSMP) provides a high-level statement of the maintenance objectives and activities for the [Operator]’s [type 2] fleet as it relates to modification upgrade and refurbishment programs as they arise will be managed through the same systems described.

In summary we have operational systems/processes in place underpinned by railcar system design, in-house training, qualified staff , structured maintenance programs, approved part suppliers to promote critical system safety. 

ATSB comment

The ATSB invited the the maintenance contractor to provide a further response to specifically address the reporter's concern relating to the absence of operational procedures for the verification of the serviceability of critical train systems such as Automatic Train Protection (ATP) by qualified engineering staff before the train is returned to service following maintenance or damage repairs. The maintenance contractor provided a further response as follows:

Our staff are trained in the procedure/s we have previously provided and more; the critical systems are designed to perform a health check function this forms part of any repair process; 'check the system is operational post repair', the work order or fault booking is then completed, if the fault remains the railcar is held until the repair is complete, that’s not to say the system may not have another fault after the repair and successful health check. 

All faults are recorded in our Computerised Maintenance Management System (CMMS), this is also fully accessible to the asset owner and operator ([Operator]).

On completion of the repair, the fault log (Driver Booking or Work Order) is processed by our maintenance team and asset team and completed in the CMMS. The railcar is released into service after repairs are completed, this is a process (procedure).

If the concern raised relates to in-service operations (operational procedures; passenger carrying) then this is not a maintenance function and sits outside the maintenance processes/procedures and needs to be addressed with the railcar operator, [Operator].

Each repair doesn’t necessarily have a specific procedure, the procedure is as defined in the faulty management process/service system manuals (a copy previously provided) and local specific procedures and RSMP previously provided, the maintainer will use their inherent trade skills, maintenance service manuals, and specific procedures as applicable to the repair being performed.

Rolling-stock operator response 

[Operator] notes that ATSB has provided clarification that the REPCON report is with reference to [Operator]’s [type 2] electric multiple unit (EMU) fleet. [Operator] engages a contractor ([Contract company]) to carry out maintenance for our [type 2] EMU fleet. We are aware that [Contract company]) has also been contacted for comment with respect to the REPCON report. Therefore, this response will be limited to the procedures applicable to the [type 2] EMU fleet, and while we may refer to the contractor’s documented procedures, we have not included copies of those documents with this response.

[Operator] acknowledges that the serviceability of safety critical systems should be verified following maintenance that potentially compromises those systems. [Operator]’s view is that verification should be completed by staff who are competent in the specific verification tasks. Systems are generally designed such that foreseeable failure modes and corresponding maintenance activities generally don’t require a qualified engineer to sign-off. For example, a design incorporating exchangeable sealed modules (which the ATP system uses extensively) is intended to allow repair by staff who only need to know how to exchange the module and verify a limited number of indicator statuses. Engineers involved in system design pre-determine which activities and tasks can be safely completed by less qualified staff. Ideally, all foreseeable maintenance tasks could be completed with the least skill requirements possible. This is a principle of design for maintainability.

Safety critical systems often include fail-safe design elements, to narrow down the scope of truly safety critical elements in a system where a plausible failure could cause a wrong-side consequence. In a well-designed system, safety critical function verification occurs automatically at start-up and continuously throughout operation and requires no special skills beyond standard operator training. The ATP system does this to a degree. Finally, operating procedures are developed to ensure safety so far is as is reasonably practicable even in the event of a system failure during service.

To summarise, safety critical systems that are well designed for maintainability and safety have expected failure conditions with pre-defined appropriate maintenance and operational responses and are designed in such a way as to minimise the qualification requirements for functional verification. The maintenance responses and required verification for the most common expected failure modes are designed to be completed by operators or front-line maintenance staff. The repair of elements for which it is not reasonable to have competent staff on site, often require components to be sent back to the Original Equipment Manufacturer (OEM) for repair or replacement. For example, repair to internal components of ATP modules. Parts repaired off-site by the OEM are required to be functionally verified and certified, and documented evidence to this effect is provided, as part of the scope of work.

Regarding the implication of the REPCON report that a specific safety critical system verification is required as direct consequence of all instances of maintenance activity on a train, [Operator] observes that the nature of a serviceability verification task is dependent on the context, that is, has an event occurred that could plausibly compromise the function of a safety critical system, and if so, in what way? This means that;

With the above said, the scope of maintenance that can safely be completed at the front-line needs to be documented, and staff completing the tasks need to be assessed to be competent. The processes under which front-line maintenance staff carry out and record work and verify system function should be documented.

A front-line maintenance organisation will often create documentation that augments, copies, or references documentation passed on from the OEM or the engineering authority, while an operator or engineering authority in turn would document requirements that constrain and oversee the activities of the front-line maintenance organisation.

Therefore, the '…formalised engineering verification processes on critical train safety systems…' referenced by the REPCON report may be considered to include documented requirements throughout the chain of responsibility, including the equipment manufacturer, systems integrator, operator, engineering authority and maintenance organisations. It is infeasible for us to cover all elements of assurance across all organisations, fleets and safety critical systems in this response.

Instead, this response will focus on detailing the documented process that apply:

Generally, these processes shall comprise a staff member verifying the state of an indicator of function they are competent to recognise.

It should be noted that for reasons of efficiency, the procedures that are used to verify satisfactory safety critical function are often the same procedures that are used to verify other operationally critical but not necessarily safety critical functions.

Routine Operation Checks

Safety critical system designs often include features which allow basic serviceability to be verified by the operator. For example, observing headlight function, testing the horn, testing brake application and so on. In the case of the ATP system of the [type 2] EMUs, system functions are automatically tested in a Cab Activation Startup Test, and brakes cannot be released if these self-tests are not passed, or the system itself is bypassed.

Related [Operator] Document(s):

Extract from Service Manual (supplied to the ATSB and ONRSR)

Drivers’ Preparation

Drivers are required to perform basic functional verification checks when preparing any railcar for service that has been stabled or under maintenance. While these checks are not performed by maintenance staff, they necessarily occur prior to railcars entering service.

Related [Operator] Document(s):

[Document related to work instructions])

Maintainer-Operator Handover

Procedures for formal handover of units between the maintenance organisation and the operating organisation vary, depending on the specific relationships between the parties and the operating context.

In the case of [Operator]’s [type 2] EMU, contractual arrangements provide distinction between units under the control of the operator ([Operator]) and units under the control of [Operator]’s contracted maintainer [Contract company]. The contract applies constraints on the condition of units that may be handed over to [Operator] for operation.

The contracted maintainer has internal procedures to ensure the condition of the units at handover. Any potential fault reported by operators or maintenance staff which results in the unit being handed-over to control of the maintainer will trigger the procedures for control of maintenance work described in [Contract company] document [document 1]. This procedure includes a requirement that 'The rail vehicle shall not be released back to service if a safety critical fault has been identified and is still awaiting rectification'.

The identification and suitable rectification of faults is governed by staff operating within their qualified competency (see Maintainer Competency below) and technical documentation of the asset and its pre-defined maintenance procedures. Maintainers and their supervisors are understood to be able to recognise the scope of their competency and to be able to escalate issues to recruit appropriate resources such as the maintainer’s on-site engineers, [Operator]’s engineers, or OEM support as necessary.

Maintainer Competency

The procedures described in this REPCON response rely on supporting management systems to ensure that staff carrying out defined roles have the appropriate level of technical competence.

For example, in [Contract company] document [document 1], certain tasks are to be carried out by roles such as Maintenance Technician, Maintenance Supervisor or Engineering Manager. These key roles require specific competencies for assurance of accurate fault diagnosis, rectification action, and functional verification activities.

[Operator] understands that [Contract company] has a comprehensive staff competency management system including positional descriptions, recruitment criteria, training matrices, and so on. Maintainer training includes formal in-house training in the systems of the [type 2] EMU trains.

Preventative (Routine) Maintenance

Preventative maintenance is the periodic completion of condition checks, part renewals, and other maintenance tasks defined in documented maintenance plans and manuals for a specific rollingstock configuration.

The tasks described for a preventative or routine maintenance event may include instructions aimed to verify system function prior to the return to service. In the case of the [type 2] EMUs, [Operator] owns and controls asset information including the Service Manual, which contains the routine maintenance schedule and OEM maintenance instructions, while [Operator]’s contracted maintainer [Contract company] utilises this information to develop local procedures specific to their maintenance operation.

Related [Operator] Document(s):

Extract from Service Manual (supplied to the ATSB and ONRSR)

Corrective Maintenance (breakdown, damage repairs, and so on)

Maintenance requirements arising from breakdown or incidental damage may involve an expected failure mode of a system or may involve a previously unanticipated failure mode. However, systems (including documentation and maintenance arrangements) are designed so that, regardless of the root cause of an abnormal condition, functional failures are classified, causes diagnosed, and appropriate responses utilised.

If an expected failure mode occurs, the pre-determined maintenance response defined in system documentation may be used, including any specified functional verification.

In the event of an abnormal condition not covered in standard documentation, the work required to restore nominal configuration and condition must be assessed, inclusive of requirements for functional verification. This activity must be conducted by staff who are competent in technical understanding of the relevant system/s and the application of assurance processes.

Depending on the complexity of the system being maintained and its abnormal condition, this type of rectification and verification may be planned by Maintenance Technicians or may involve qualified engineers. Escalation of complex faults is noted within [Contract company] document [document 1].

Closing Comments

[Operator] is satisfied that formal procedures are in place via the maintenance contractor to ensure that [type 2 train] safety critical system serviceability is verified following maintenance events. The nature of these verification activities depends on the nature of the maintenance activity, and/or information known or suspected regarding the condition of the rolling stock. In addition, many safety critical systems feature routine functional verification as an operator check before and during operations.

Despite these efforts, a system may present as fully functional when tested and subsequently fail in service. It is not reasonably practicable to eliminate in-service breakdowns. However, the maintenance organisation is required to minimise the rate to an acceptable level through performance agreements.

Finally, safety critical systems are designed to be fail-safe so far as practicable, and operational controls are in place to mitigate the risk profile of operation after an in-service failure but before returning a railcar for maintenance.