Refine search

Reports

13 December 2017

Published

Managing demand for ambulance services 2017

Health

Information technology

Management and administration

Risk

Service delivery

Shared services and collaboration

Workforce and capability

NSW Ambulance has introduced several initiatives over the past decade to better manage the number of unnecessary ambulance responses and transports to hospital emergency departments. However, there is no overall strategy to guide the development of these initiatives nor do NSW Ambulance's data systems properly monitor their impact. As a result, the Audit Office was unable to assess whether NSW Ambulance's approach to managing demand is improving the efficiency of ambulance services.

Demand for ambulance services is increasing. Demographic factors including population growth and ageing have contributed to this and ongoing growth in demand is likely. It is important that NSW Ambulance finds ways to respond to this demand more efficiently, while maintaining patient safety standards and meeting community expectations.

Most triple zero calls to NSW Ambulance do not involve medical issues that require an emergency response. NSW Ambulance has introduced a range of initiatives to change the way it manages these less urgent requests for assistance. Its major demand management initiatives include using a telephone advice line, referring some patients to services other than hospital emergency departments and using specialist paramedics to respond to less urgent cases.

The role of NSW Ambulance has changed in recent years. It is aiming to become a ‘mobile health service’ that identifies the needs of patients and provides or refers them to the most appropriate type of care. This change involves a significant expansion of the clinical decision-making role of paramedics. Considerable strategic and organisational efforts are required to make this work. The successful implementation of demand management initiatives is important to NSW Ambulance's ability to continue to meet demand for its services.

This audit assessed NSW Ambulance's major demand management initiatives that aim to reduce unnecessary demand for ambulance responses and unnecessary transport to hospital emergency departments. It aimed to assess the extent to which these initiatives have improved the efficiency of its services.

Conclusion

NSW Ambulance has introduced several initiatives that aim to manage demand for its services from less urgent cases more efficiently. There is no overall strategy for these initiatives and NSW Ambulance’s data systems do not measure their outputs or outcomes. As a result, we are unable to assess the impact of NSW Ambulance's demand management initiatives on the efficiency of ambulance services. More focus is needed to ensure these initiatives achieve the efficiency improvements necessary to help NSW Ambulance meet future increases in demand.

Increasing demand for ambulance services is a key issue for NSW Ambulance. Demand has increased at a faster rate than population growth in recent years and continued growth is expected. NSW Ambulance has introduced several initiatives that aim to manage demand for its services from people with less urgent medical issues more efficiently and align its approach with the rest of the health system in New South Wales.

These individual initiatives lack a broader strategy to guide their development. NSW Ambulance’s demand management initiatives also lack clear goals and performance targets, with insufficient organisational resources allocated to support their implementation. NSW Ambulance does not have a data system that allows it to conduct accurate routine monitoring of the activity and performance of these initiatives.

More effort is required to make demand management initiatives a core part of NSW Ambulance's work. Key relationships with other health services to support demand management initiatives have only recently been established. NSW Ambulance has not communicated proactively with the public about its demand management initiatives. To ensure paramedics are as well prepared as possible for their expanded roles, they need better professional development and up to date technology.

Demand for ambulance services in New South Wales is increasing steadily. Forecast future increases in demand due to population growth and ageing mean that NSW Ambulance must improve its efficiency to maintain its performance.

Demand for ambulance services is growing at a rate higher than population growth. The increase in demand is likely to continue as the population continues to grow and age. NSW Ambulance has made several recent changes to remove large parts of demand for its services, including moving non-emergency patient transport to a separate government agency and changing the way triple zero calls are categorised.

These changes were expected to improve emergency response time performance, but the anticipated improvements have not been achieved. If demand continues to increase as forecast, NSW Ambulance will need to find more efficient ways to manage demand to maintain its performance.

NSW Ambulance has introduced initiatives to change the way it manages demand from patients who have less urgent medical issues. These have the potential to achieve positive results, but we were unable to fully assess their impact because of weaknesses in data systems and monitoring. More needs to be done to demonstrate progress toward the efficiency improvements required.

NSW Ambulance uses a telephone referral system to manage triple zero calls from people with medical issues that do not require an ambulance. This has the potential to achieve efficiency improvements but there are weaknesses in NSW Ambulance's use and monitoring of this system. Paramedics are now able to make decisions about whether patients need transport to a hospital emergency department. NSW Ambulance does not routinely measure or monitor the decisions paramedics make, so it does not know whether these decisions are improving efficiency. Extended Care Paramedics who have additional skills in diagnosing and treating patients with less urgent medical issues were introduced in 2007. NSW Ambulance analysis indicates that these paramedics have the potential to improve efficiency, but have not been used as effectively as possible.

Our 2013 audit of NSW Ambulance found that accurate monitoring of activity and performance was not being conducted. More than four years later, this remains the case.

NSW Ambulance has recognised the need to change the way it manages demand and has developed initiatives that have the potential to improve efficiency. However, there are significant weaknesses in the strategy for and implementation of its demand management initiatives.

NSW Ambulance has identified the goal of moving from an emergency transport provider to a mobile health service and developed several initiatives to support this. Its demand management initiatives have the potential to contribute to the broader policy directions for the health system in New South Wales. However, there is no clear overall strategy guiding these initiatives and their implementation has been poor.

NSW Ambulance's reasons for changing its approach to demand management have not been communicated proactively to the community. Demand management initiatives that have been operating for over a decade still do not have clear performance measures or targets. Project management of new initiatives has been inadequate, with insufficient organisational resources to oversee them and inadequate engagement with other healthcare providers.

NSW Ambulance uses an in-house Vocational Education and Training course to recruit some paramedics, as well as recruiting paramedics who have completed a university degree. No other Australian ambulance services continue to provide their own Vocational Education and Training qualifications. Paramedics will need more support in several key areas to be able to fulfil their expanded roles in providing a mobile health service. Performance and development systems for paramedics are not used effectively. Up to date technology would help paramedics make better decisions and improve NSW Ambulance's ability to monitor demand management activity.

There are gaps in NSW Ambulance's oversight of the risks of some of the initiatives it has introduced, particularly its lack of information on the outcomes for patients who are not transported to hospital. Weaknesses in the way NSW Ambulance uses its data limit its ability to properly assess the risks of the demand management initiatives it has introduced.

Parliamentary reference - Report number #295 - released 13 December 2017

15 June 2017

Published

Sydney Road Maintenance Contracts

Transport

Infrastructure

Internal controls and governance

Management and administration

Procurement

Project management

In November 2013, Roads and Maritime Services (RMS) outsourced the maintenance of State roads in the Sydney region south and west zones using an innovative contracting approach called the Stewardship Maintenance Contract (SMC). The SMC links risk to reward, and uses a performance framework where outcomes should drive improved performance over time.

Roads and Maritime Services (RMS) is responsible for the Sydney region State roads network This includes over 2,800 kilometres of roads and associated road corridor infrastructure such as bridges, tunnels and drainage structures. RMS divides the network into three geographical areas: south, west and north zones.

In 1995, RMS first outsourced road corridor infrastructure maintenance for the north zone through a Performance Specified Maintenance Contract (PSMC). The current 10-year PSMC for the north zone will expire in October 2018. Prior to November 2013, RMS maintained roads in the south and west zones through its Road and Fleet Services unit.

In November 2013, RMS outsourced road maintenance services for the south and west zones using Stewardship Maintenance Contracts (SMC). The contracts run for seven years with an option for a further three years at RMS’ discretion. RMS estimated that the annual cost of these contracts was around $240 million in total. In March 2018, the contract prices are due to be reset by negotiation to reflect the contractors’ experience with, and better information about, the road networks and routine maintenance requirements.

The SMC model adopted stewardship principles to improve value for money. RMS defined stewardship principles as a broad set of values, attitudes and behaviours, required of the contractor to effectively manage the assets on behalf of RMS. The SMC also includes commercial principles, such as linking risk to reward, and a performance framework where outcomes drive performance.

This audit assessed whether RMS had effectively managed the outsourcing of road maintenance in the Sydney region south and west zones. In making this assessment, we answered the following questions:

Did RMS justify the decision to adopt the SMC model?
Do SMCs include key performance indicators (KPIs) and incentives which promote efficiency and effectiveness?
Does RMS collect high quality information on contractor performance and take action to correct performance deficiencies?
Are the expected benefits being achieved?

Conclusion

RMS developed an innovative contracting approach with the SMC. RMS has realised some benefits in the first year, including savings, from outsourcing road maintenance in the Sydney region south and west zones using the SMC. However, RMS’ management of the SMC has key elements missing which reduces its effectiveness.

The SMC includes performance measures and incentives to drive efficiency and effectiveness improvements over time.

RMS has established a contract management framework which includes most elements of good practice, including governance and dispute resolution mechanisms. However, it does not have procedures to guide its contract managers in managing specific provisions of the SMC. Consequently, RMS has not exercised several significant SMC requirements, such as having the contractor account for an efficiency dividend in its pricing at the start of each three-year works period. It also has not done enough to assure itself that the contractor provided performance and financial data are correct. This is important because the data is used to measure performance and calculate contractor payments.

RMS assessed that it had achieved around 80 per cent of the expected cost benefit in the initial year of the SMC. However, it has not tracked its achievement of benefits since then.

The Stewardship Maintenance Contract

RMS justified adopting the SMC model and included KPIs to drive efficiency and effectiveness

The SMC model includes features that RMS had not previously used for road maintenance contracts. These included adopting stewardship principles and transferring price risk to the contractor over time as the contractor becomes familiar with the assets being maintained.

The SMC model meets RMS’ requirements for flexibility in pricing models, the need for collaboration in asset maintenance planning, promoting innovation and effective performance management.

RMS used many good practices to develop the SMC model, including:

preparing a robust business case comparing the SMC model to RMS maintaining the road network itself, as well as assessing whether two other contracting models
(traditional and alliance) would meet its requirements
assessing experiences with similar arrangements in other jurisdictions and identifying elements that worked to get the best outcomes
developing a robust performance framework, which included a mix of efficiency and effectiveness KPIs that reflected NSW Government policy and RMS priorities
incorporating risk and reward incentives delivered through cost sharing arrangements which change as the contract matures
using a contract duration that supports RMS priorities and provides an incentive for better quality outcomes.

RMS uses data provided by the contractor to measure performance and calculate payments to the contractor. The SMC includes a specific sanction if RMS finds that the contractor provided incorrect performance data, but no specific sanction if the contractor provides incorrect financial data. If RMS finds that the contactor provided incorrect performance or financial data, RMS can only recover over-payments which may have been made using the incorrect data.

To provide a stronger incentive for the contractor to ensure data it provides is accurate, RMS should consider whether to incorporate stronger sanctions when negotiating the commercial reset due in mid-2018 for south and west zones. RMS should also consider this for the new contract for the north zone when the current PSMC contract expires in October 2018.

RMS' contract management approach and benefits realization

RMS can improve the effectiveness of its oversight and management of the SMC

RMS does not have SMC specific contract procedures to guide its contract managers. Consequently, RMS has not exercised several significant SMC requirements, such as having the contractors account for an efficiency dividend in their pricing at the start of each three-year works period. Effective contract management should be supported by contract specific procedures, with explanations of, and allocation of responsibility for, the various interventions that RMS may be required to exercise in the SMC.

Performance and financial reporting under the SMC is based on a mix of RMS and contractor provided data. While there are a range of audits of contractor provided performance and financial data that RMS can conduct each year under the SMC, it does not have a schedule of audits it will conduct and when.
During the first year of the SMC, RMS commissioned some limited audits of financial data. In the first three years of the SMC, RMS did not conduct any audits of performance data. Had there been SMC specific procedures in place, this would have reduced the risk of RMS not implementing a systematic audit program to give it reasonable assurance on the quality of the data that the contractor has provided. This is important because the data is used to measure performance and calculate contractor payments.

RMS has been aware of data quality issues since 2015. While RMS advised that it commenced addressing some data quality issues in response to a series of reviews conducted in 2015, a recent internal audit report indicates that RMS has not resolved the data quality issues.

RMS achieved benefits in the first year, but has not tracked benefits since

As part of the business case, RMS agreed to implement a benefits realisation strategy, including a benefits tracking tool. RMS commenced tracking benefits, but did not establish a comparative baseline pre-SMC on non-financial benefits, and has not tracked benefits past year one.

In 2015, a benchmarking study commissioned by RMS found that it had achieved 80 per cent of the expected recurrent cost savings and other benefits, such as improved workplace safety, in the first full year of the SMC. However, there was no clear baseline to measure
non-financial performance. The study was qualified due to gaps in available data. The study also did not reconcile the actual one-off transition costs to the business case estimate.

During the course of the audit, RMS advised that it intends to repeat this type of study to determine whether it has achieved all expected benefits (and their value), and that it would use the results to inform its negotiation with the SMC contractors as part of the commercial reset due in mid-2018.

Roads and Maritime Services is responsible for the State Roads network in the Sydney region

Roads and Maritime Services (RMS) is responsible for the Sydney region State roads network. This includes over 2,800 kilometres of roads and associated road corridor infrastructure such as bridges, tunnels and drainage structures. The network is divided into three geographical areas: south, west and north zones. Prior to November 2013, RMS maintained roads in the Sydney region south and west zones through its Road and Fleet Services unit.

In 1995, RMS first outsourced road corridor infrastructure maintenance for the north zone through a Performance Specified Maintenance Contract (PSMC). The current 10-year PSMC for the north zone will expire in October 2018. This contract is worth around $35 million per annum.

NSW Government priorities and road maintenance

Efficient and effective road maintenance contributes to the following NSW Government priorities:

improving road travel reliability
ensuring on-time running of public transport
reducing road fatalities
improving government services
keeping our environment clean.

The NSW Commission of Audit recommended outsourcing the maintenance of State roads

The NSW Commission of Audit in its Final Report on Government Expenditure (May 2012) recommended contestability as an appropriate strategy to consider for improving road maintenance service delivery for State roads.

The Commission benchmarked RMS’ road surface quality and cost per lane kilometre against those of Western Australia, Victoria, and Queensland. This showed that New South Wales lagged the other states on both these measures.

Exhibit 1: Interjurisdictional comparison of road maintenance outcomes 2009–10
	WA	VIC	QLD	NSW
Roads managed (lane kms)	52,659	50,510	71,353	80,348
Estimated spend ($/lane km)	5,000	4,500	6,000	7,000
Road quality measure (%)	99	99	94	91

Source: NSW Commission of Audit Final Report May 2012.

The Commission noted that RMS had conducted two independent reviews to examine the potential for extending road maintenance contestability. The Commission found that there was inadequate and inconclusive benchmarking to establish the efficiency of RMS’ Road and Fleet Services unit when compared to outsourcing. It recommended that RMS bring forward a proposal to conduct a competitive tender for the road maintenance of the Sydney region south zone road network to inform the feasibility of a progressive rollout of road maintenance contestability across other areas of the State. In August 2012, the NSW Government adopted the Commission’s recommendation.

The NSW Government introduced road maintenance contestability through Stewardship Maintenance Contracts

In April 2013, the NSW Government announced that it would introduce road maintenance contestability across the Sydney region, using a Stewardship Maintenance Contract (SMC) model to improve value for money. In doing so, it excluded RMS’ Road and Fleet Services unit from tendering.

The SMC model is based on the following key commercial and performance principles set by RMS:

performance driven by outcomes
flexible and adaptable
transparent and measurable
linking risk to reward
continuous improvement
criteria for selection of, and transition to, different payment models.

The following key stewardship principles underpin the SMC’s broad set of values, attitudes and behaviours, which are required of the contractor to effectively manage the assets on behalf of RMS:

putting RMS’ customers (road users and the general public) first and being responsive to them
being responsible and accountable for the outcomes resulting from the management of the assets
managing the assets diligently, efficiently and effectively with limited direction from RMS
working collaboratively with RMS to deliver services that are tailored to meet RMS’ evolving needs
acting with integrity and transparency in performing the services
performing the services in the best interests of RMS and asset users.

Other key features of the SMC include:

service requirements which describe the scope of the services, and the standards the contractor must meet
a commercial framework which defines how payments are structured, how performance assessment will impact on payments and outlines the key commercial principles. SMCs primarily divide payments into two main mechanisms, these being the priced component (or fixed price) and the target cost calculated as follows:
- fixed price – the contractor is paid a pre-agreed amount for specific services being provided, regardless of the actual costs incurred
- target cost – RMS and the contractor agree on a target cost for a project, and any cost overruns or underruns are shared between them
a performance framework which provides mechanisms for assessing contractor performance. This includes a comprehensive listing of the key result areas (KRAs) and key performance indicators (KPIs) against which RMS measures the contractor’s performance. The framework also outlines the scoring methodology that RMS uses to determine whether the contractor’s bid margin (profit and overheads) is reduced due to less than satisfactory performance or whether a bonus is paid if a threshold performance score is exceeded.

Road maintenance under SMCs for Sydney region south and west zones commenced in November 2013

In November 2013, RMS awarded SMCs to the Leighton Boral Amey consortium, now named Ventia Boral Amey (VBA), for the south zone and the DownerMouchel (DM) consortium for the west zone. The contracts run for seven years with an option for a further three years at RMS’ discretion. In April 2014, full services commenced following a four-month transition period. RMS estimated that the annual cost of these contracts was around $240 million in total. In March 2018, the contract prices are due to be reset by negotiation to reflect the contractors’ experience with, and better information about, the road networks and routine maintenance requirements.

Roads and Maritime Services should consider whether to incorporate stronger sanctions in the Stewardship Maintenance Contract if the contractor provides incorrect performance or financial data to RMS, when:
1. negotiating the commercial reset for the next works period with the Sydney region south and west zone contractors due in July 2018.
2. finalising a new SMC contract for the Sydney region north zone, due to commence in October 2018.

Roads and Maritime Services should, by September 2017:

2. Review its contract management framework for SMCs to ensure that all authorities and accountabilities of
contract managers are clearly defined, including:

a) accountability and procedures for exercising all operational clauses in the SMC where RMS may opt to, or be required to intervene, or make a decision

b) authority to approve or initiate the interventions RMS is required to, or may, exercise under the SMC

c) the audits that RMS will conduct to systematically validate the performance and financial data that the SMC contractors provide

d) the accountabilities of RMS contract managers to systematically review audits and quality reviews that the SMC contractors must conduct to demonstrate compliance with their service plans

e) the accountabilities of RMS contract managers to check that the monthly and annual reports provided by SMC contractors do not contain errors, omissions or inaccuracies.

3. Improve its management of benefits realisation by:

a) initiating a further benefits realisation review and record the benefits delivered against those
estimated following the tender process, including the one-off transition costs

b) identify any benefits, including savings, not yet attained and develop strategies to address any short-falls

c) establish a tool to track the ongoing realisation of benefits.