Executive summary

	The focus of our audit
	One of RailCorp’s key performance targets is for 92 per cent of peak hour services to run on time. Signalling failures can delay many trains and inconvenience many passengers. Effective maintenance and response to signalling failures is therefore crucial to achieving on-time running.
	Sydney’s rail network is large and complex. The signalling system has many parts, including around 3,800 trackside signals, 2,900 train stops, 6,800 track circuits and 2,200 sets of points. The system employs a range of technologies, some quite old.
	This audit examines whether RailCorp is effectively managing the risk of signal failures. It focuses on whether RailCorp is keeping the number and duration of signal failures low enough to support its on-time running target.
	A train is defined as on-time if it arrives at its final destination within five minutes of schedule if a suburban train (four minutes prior to June 2005), and six minutes if an intercity train. If a train is not on-time, it is deemed to be delayed. A signalling incident occurs when a signalling failure causes at least one peak train to be delayed.
	Peak services are those scheduled to arrive in the Sydney CBD between the hours of 6 am and 9 am (morning peak), and departing the Sydney CBD between the hours of 4 pm and 6 pm (evening peak) from Monday to Friday.
	The NSW State Plan released in September 2006 targets a substantial increase in the use of public transport by 2016. RailCorp is due to make a submission to government later this year outlining what will be required of the network, including signalling, to allow it to meet patronage increases.
	Audit opinion
	Between 2004 and 2006, the number of signalling failures, signalling downtime and the number of trains delayed as a result of signal failures all fell. RailCorp’s on-time running performance improved over the same period.
	The fall in failures is a clear indication of improved performance. Changes in the definition of on-time and to the timetable during 2005 and 2006 however make it difficult to determine whether improvements in response downtime and signalling delays are due to a true performance improvement.
	RailCorp has improved its management of signal assets and its response to signal incidents. For example, it now has an incident response framework and places key staff at critical locations to respond quickly to signal failures. This has contributed to these improved results.
	To build upon this strong base, RailCorp needs to determine with more confidence the number and duration of signalling failures the network can tolerate without impacting on service levels. This would reduce the risk that it may focus too little or too much effort on signalling failures compared to other causes of delays such as train breakdowns.
	RailCorp also needs to compare its signalling reliability and incident response practices and performance against other networks. This would give it another important perspective on its performance.
	It needs to determine the signalling system required to meet the expected 2016 patronage levels, and how it will get there. This would reduce the risk of spending money on a signalling asset which will not meet projected service demands.
	Over the next few years, RailCorp will also need to be vigilant to ensure maintenance does not lapse. Much work to upgrade and expand the network is planned, such as Rail Clearways and the new north-west and south-west rail lines. These projects will place pressure on RailCorp’s maintenance spending and its pool of skilled staff.
	Key audit findings
Is the number of signal failures low enough?	Between 2004 and 2006: § the number of 24 hour signalling failures on the greater metropolitan network fell by nine per cent § the number of signalling incidents fell by 45 per cent § RailCorp met its monthly signalling failure targets most of the time. Railcorp signalling asset management practices have improved. For example, RailCorp has adopted a good asset management model, improved its collection and analysis of data, and is on target to eliminate by 2011 the backlog which occurred in the late 1990’s and early 2000’s due to inadequate funding. The improvement in 24 hour signalling failures indicates that asset performance has improved. RailCorp was not, however, able to say by how much the fall in peak incidents resulted from changes to the timetable in 2005. We could not conclude whether RailCorp was keeping signalling failures low enough. It uses past performance to determine the number of signalling failures the network can tolerate while still retaining service levels, but should also trial reliability modelling. RailCorp has yet to adequately compare the reliability of its signalling system to other networks. RailCorp is working to determine the signalling system required to meet the State Plan’s 2016 patronage targets. Without this, it could spend money on upgrading signal assets which may be replaced or removed from service shortly thereafter. Much work on expanding and improving the network is planned over the next few years. This could put pressure on signal maintenance spending and staffing. In particular, RailCorp and the rail industry is facing a worldwide shortage of signal engineers and electricians.
Recommendations	We recommend that RailCorp: § use both past performance and reliability modelling to estimate the number of signal failures the network can tolerate and set targets accordingly (page 15) § determine the signalling system it needs to meet the government’s 2016 patronage target as soon as possible, and documents by the end of 2008 how it intends to get there (page 20)
	§ ensures that it balances the resource demands of maintaining the existing network and eliminating the backlog against those arising from planned network expansion and improvement projects (page 22) § where possible redesign work practices to help address the emerging shortage of signal engineers and electricians (page 22) §  benchmark the reliability of its signalling assets against other railway operators with similar operating environments (page 23).
Is the duration of signal failures low enough?	Between 2004 and 2006 the average: §   number of peak trains delayed per month due specifically to signalling failure fell by 64 per cent (280 to 102) § signalling downtime per month fell by about 17 percent. RailCorp’s incident response procedures and practices have improved. RailCorp has implemented an incident management framework. It locates key staff across the network to respond to signalling failures, and during peak times places staff on standby at vulnerable parts of the network. RailCorp is not able to determine how much the fall in delays resulted from changes in the definition of a delayed train and to the timetable. We could not conclude whether RailCorp was keeping the number of delays and the duration of signalling failures low enough. It uses past performance to determine what the network can tolerate while still retaining service levels, but should also trial reliability modelling. In relation to signalling, RailCorp is yet to review how well its incident response framework is implemented, whether it results in satisfactory outcomes and how it captures lessons for continuous improvement. While it locates response staff around the network, it needs to undertake a systematic risk assessment to ensure it has the right people, at the right place, at the right time. It also needs to do more to ensure staff possess the skills to respond well to signal failures. RailCorp is yet to benchmark its response performance with other networks. It reports its performance in regard to on-time running and delays that occur during the peak hours, but should also report this on a 24 hour basis.
	We recommend that RailCorp: § use both past performance and reliability modelling to estimate the duration of signalling delays the network can tolerate and set targets accordingly (page 29) § review by the end of 2008 how its incident response framework impacts on signal incidents (page 31) § base incident response strategies on a systematic risk assessment (page 33) § review competencies of staff involved in signal asset management or incident response by the end of 2008 and address skills gaps (page 33) § benchmark incident response against other railway operators with similar operating environments (page 35) § implement in 2007-08 its plans to move to 24 hour on-time running reporting (page 35) § monitor and report on asset performance and its impact on on-time running on a regular basis (page 35).
	Response from RailCorp
	I am pleased to have received the Final Report from the Audit Office on “Performance Audit, Signal Failures on the Metropolitan Rail Network” for 2007.
	The fact that delays relating to signalling equipment have decreased by 9 per cent over a 24 hour period, and 45 per cent during peak times, highlights the work done in this important area.
	RailCorp has invested much time and expertise in providing solutions that limit the number of signal related incidents on the rail network.
	By reducing the total number of signal issues on our infrastructure, our passengers benefit from a more reliable service and face less delays.
	The improvements to our signalling system have contributed to CityRail meeting its on-time running benchmark of 92% for 2006/07.
	I also note that we are encouraged by six of our initiatives being highlighted as good practice in the report, including:
	(i) initiatives to address vandalism; (ii) asset management process initiatives; (iii) training initiatives such as the clawlock points example to improve reliability; (iv) initiatives to improve incident response; (v) continuous improvement initiatives aimed at improving skills of staff responsible for signalling maintenance and failure response; and (vi) initiatives aimed at monitoring asset and response performance which include many forums and detailed reports.
	Whilst RailCorp has demonstrated that signalling performance has improved, we acknowledge that further work is still required “to build upon this strong base” to continually improve the service that is provided to our customers.
	It is also pleasing that the Auditor’s recommendations to achieve this outcome are consistent with RailCorp’s designated objectives and future direction. In particular, the following initiatives currently being pursued by RailCorp, are focussed on meeting the common goal of improved customer service:
	(i) benchmarking our performance against other networks; (ii) increasing the use of reliability modelling throughout the network; (iii) ensuring our signalling resource demand is balanced between our maintenance and expansion projects; (iv) risk assessing our signalling incident response strategies; (v) implementing plans to move to 24-hour performance reporting; (vi) determining future signalling system requirements after the Automatic Train Protection trial; and (vii) using past signalling incidents to learn lessons and review the incident management framework.
	Once again, the outcome of the audit is a positive one and I would like to extend my thanks to the audit team for the open and professional manner in which they conducted the audit. We look forward to meeting the challenge of improving our customer service performance in the future.
	(signed) Vince Graham Chief Executive Officer Dated: 6 August 2007