Demand for ambulance services is increasing. Demographic factors including population growth and ageing have contributed to this and ongoing growth in demand is likely. It is important that NSW Ambulance finds ways to respond to this demand more efficiently, while maintaining patient safety standards and meeting community expectations.

Most triple zero calls to NSW Ambulance do not involve medical issues that require an emergency response. NSW Ambulance has introduced a range of initiatives to change the way it manages these less urgent requests for assistance. Its major demand management initiatives include using a telephone advice line, referring some patients to services other than hospital emergency departments and using specialist paramedics to respond to less urgent cases.

The role of NSW Ambulance has changed in recent years. It is aiming to become a ‘mobile health service’ that identifies the needs of patients and provides or refers them to the most appropriate type of care. This change involves a significant expansion of the clinical decision-making role of paramedics. Considerable strategic and organisational efforts are required to make this work. The successful implementation of demand management initiatives is important to NSW Ambulance's ability to continue to meet demand for its services.

This audit assessed NSW Ambulance's major demand management initiatives that aim to reduce unnecessary demand for ambulance responses and unnecessary transport to hospital emergency departments. It aimed to assess the extent to which these initiatives have improved the efficiency of its services.

Appendix one - Response from agency

Appendix two - About the audit

Appendix three - Performance auditing

The Department of Planning and Environment (Department) administers five energy rebate schemes targeted to low-income households. The five rebates are of two key types:

1. Ongoing support to pay energy bills
2. Crisis Support  

More than one million rebates are paid each year to over 800,000, or around 27 per cent, of NSW households. Households learn about rebates from a variety of sources including: Service NSW, government and energy retailer websites, energy retailer welcome packs, Department marketing efforts, information on energy bills, and Centrelink.  

The budget for energy rebates is increasing every year and in 2017–18 is more than $245 million. The Department delivers most rebates through a network of partnership arrangements with:

• energy retailers, who apply rebates directly onto energy bills
• more than 340 charities and other NGOs who assess households' eligibility for crisis support and distribute support through the Energy Accounts Payment Assistance scheme (EAPA)
• Service NSW, who informs NSW households about rebates through their call centre.

The energy rebates budget is substantial and the distribution arrangements are complex. The objective of the audit was to assess whether the current design and distribution of energy rebates schemes is effective.

The Department and partner organisations administer the rebate schemes as designed

The Department oversees a complex package of rebate schemes in partnership with 25 retailers and around 340 charities and NGOs. The partnership arrangements ensure that funds are distributed directly to energy bills as intended. The schemes provide support to recipients and are administered in line with government decisions about eligibility.  

Communication about rebates does not reach all eligible households

Households learn about rebate schemes through a mix of communication channels including retailer websites and call centres, Department websites, Centrelink, financial counsellors, EAPA Providers, the Energy and Water Ombudsman and Service NSW. Some low-income groups, such as those with poor English language skills, do not find out about energy rebates.

Scheme objectives are not measurable

Rebate schemes that provide ongoing support do not have measurable objectives or outcome measures. Without clear and measurable objectives, the Department cannot report to government on whether the schemes are achieving the intended policy outcomes, nor recommend improvements to ensure the schemes deliver the greatest benefit to the most financially vulnerable households.

The EAPA crisis support scheme has a clearer objective in that it aims to keep households experiencing financial crisis connected to energy services. However, the Department does not measure outcomes from providing this type of support, and does not know if the crisis support achieves this objective.  

The structure of rebate schemes for ongoing support is complex

The Low Income Household Rebate accounts for 80 per cent of the budget for ongoing support rebates. The remaining 20 per cent of the budget is administered through four separate schemes: Gas Rebate, Medical Energy Rebate, Family Energy Rebate and Life Support Rebate.

Each of these rebates has its own eligibility criteria and some require separate application processes. The Family Energy Rebate is complex to access and apply for, and around one third of households do not reapply each year. Eligible households that receive energy through embedded networks apply directly to the Department for rebates, which are paid by the Department into bank accounts. Embedded networks are energy supply arrangements where the manager of a residential facility such as a caravan park, retirement village or apartment block, buys energy in bulk and then on-sells it to residents. The Department is yet to develop strategies to address a forecast increase in such households.

The design of the rebate schemes creates some inequities

Households in similar circumstances can receive different levels of assistance depending on which adult in the house is the energy account holder, the mix of energy types used in the home, or the EAPA Provider they turn to when in financial crisis.

Households with both gas and electricity connections receive more assistance than those with only electricity. Households in rural and regional areas receive the same value rebate as households closer to Sydney, despite higher distribution charges. Family Energy Rebate is a two-tier payment, with a higher amount available to families with greater means. Lower-income families receive a much smaller Family Energy Rebate on the assumption that they already receive Low Income Household Rebate. Charities and NGOs distributing EAPA crisis support apply inconsistent standards when assessing household need, which leads to inequitable levels of assistance.

Departmental oversight of energy retailers and EAPA Providers is not strong enough

While partnering with energy retailers and EAPA Providers delivers advantages, stronger management is needed to ensure that partners follow Departmental guidelines and to minimise the potential for fraud. The Department's accreditation process for potential EAPA Providers does not consider the applicant's financial governance standards and the most recent audit of EAPA Providers was 2013.

Appendix one - Response from the Agency

Appendix two - About the audit

Palliative care is an essential component of modern health care services and an increasingly important part of the wider health and social care systems. Palliative care is healthcare and support for people with a life-limiting illness, their families and carers. It is provided by, or informed by, professionals who specialise in palliative care. ‘End of life’ care is provided to people approaching the end of life by health professionals, who may work in the health, community or aged care systems. Not everyone receiving end of life care needs palliative care.

NSW Health has a policy and planning role in palliative and end-of-life care, and it coordinates a wide range of service providers. Local Health Districts (LHDs) provide care services in settings such as homes, hospitals and clinics to patients with varying needs. There are several care providers that can be involved.

Due to this shared nature of palliative care — where many people, services and settings are involved in delivering care to the patient — availability and communication of information is critical. For service planning, data and evidence must be drawn from various sources in a timely and efficient way.

This audit assessed whether NSW Health is effectively planning and evaluating palliative care services, in the context of rising demand, increasingly complex needs, and the diversity of service providers.

1. By July 2018, NSW Health should develop an integrated palliative and end-of-life care policy framework that:

• clearly articulates the interface between palliative and end of life care and outlines the priorities for the respective areas
• defines policy goals and objectives, and a performance and evaluation framework for palliative care service planning and delivery
• informs a related workforce plan which supports the policy framework and is linked to the Health Professional Workforce Plan 2012–2022
• reviews the funding allocation model to ensure future enhancement funds are distributed equitably and transparently based on the need and population of districts.

By December 2018, NSW Health should:

2. assess how the functionality provided in data collection programs such as the Palliative Care Outcomes Collaboration program can be provided across all palliative care services in NSW

3. complete its statewide review of systems and reporting for end of life management including specialist palliative care, and develop a business case to implement a more integrated set of solutions to:

• support providers delivering end of life and palliative care
• help monitor service quality and quantity
• provide comprehensive data for service planning

4. improve stakeholder engagement by:

• developing a statewide stakeholder engagement strategy that brings together current activity and good practice, and is transparent and publicly available
• defining accountability for overseeing and implementing the strategy at state and district levels.

1. Performance monitoring is inadequate

NSW Government policy on palliative care is outlined in the NSW Government Plan to Increase Access to Palliative Care 2012–2016 (the Plan). Under the Plan, the overarching policy is ‘to ensure that everyone has access to quality palliative care regardless of their economic or social circumstances, their geographical location or their medical condition.’ Some initiatives under the Plan are still being implemented.

NSW Health only has measures in place to assess some processes and activities for individual initiatives under the Plan. There is no tracking of outcomes relating to the policy goals set out in the Plan, such as increased choice to die at home or the location of the patient’s choice, and improved access to specialist palliative care services. NSW Health has not conducted an overall assessment of the Plan’s outcomes to guide future priorities.

Further, there is no overall performance and reporting framework for palliative and end of life care, meaning there is no monitoring of performance of palliative care services for NSW as a whole. This lack of evaluation and performance measurement impacts on NSW Health's ability to monitor progress and achievements, address gaps in service, and plan for future service enhancement. 

2. Statewide planning and evaluation lacks coordination

Currently, palliative care services are complex to plan and evaluate. Many policies, strategies, guidelines, directives and data collections currently inform services. Even definitions of services vary. The split of policy functions for palliative care and end-of-life care between different branches within NSW Health adds further complexity. These arrangements create the risk of confusion, gaps in advice and support for LHDs.

Consistency is needed in the use of terminology and planning to achieve an integrated approach at all levels, including:

• standard definitions of palliative care and end-of-life care
• planning within a single structured policy framework to help clarify what services are to be delivered, who is accountable for delivering them and how to measure their outcomes.

Workforce planning is also affected. While NSW Health has identified significant gaps in the specialist palliative care workforce (especially in regional and remote areas) and it previously made workforce capacity one of its priorities, limited work has been undertaken in producing a statewide strategy to reduce these gaps.

3. District planning is not systematic and some external providers are poorly managed

An integrated approach would inform district-level service planning for palliative care. Planning in the districts we visited was sometimes ad hoc and accountability for performance unclear. Districts would benefit from:

• better integrating data collection systems with planning
• clearer guidelines, easy-to-use tools, monitoring and accountability systems.

The recently developed guide – A Blueprint for Improvement, from the Agency for Clinical Innovation – should help districts plan more effectively and consistently as it rolls out more widely in 2017. This takes an integrated approach to palliative and end-of-life care. Only one district we visited has finalised a comprehensive plan using the Blueprint.

Issues with district planning extend to external agreements with service providers, as these are sometimes poorly managed and do not support improved patient outcomes. Examples we reviewed showed a significant reporting burden with process-focused reporting. We also found little evidence of monitoring or action as a result of these reports.

4. Diverse information systems mean data collection and use are inconsistent

NSW Health gathers a broad range of data from many collection points and systems to inform palliative care services at hospital, ward or unit level, and community teams. However, the current data is limited because: 

• activity is under-reported, particularly in community-based services
• collection is not universal across districts and services.

Districts also struggle with evidence-based planning and service delivery because multiple information systems mean data may be incomplete or inaccurate. Too often, clinicians and service managers rely on manual collection and paper-based systems. 

eHealth, which coordinates information communication technology (ICT) for the state’s healthcare, is planning a statewide approach to capture information and report on all palliative care activity. The current plans of eHealth to review and improve systems should make data more complete, robust and accessible for quality improvement and planning.

5. An overarching stakeholder strategy would strengthen engagement

Just as data is central to effective planning and evaluation, so too is stakeholder engagement. However, there is currently no explicit stakeholder strategy, which means consultation is inconsistent across the state and not systematic at a district level.

While NSW Health uses a range of platforms to consult, the purpose and value is often not clear to stakeholders. Individual districts have some good practices, but there are limited mechanisms to identify and share these with other areas. A state-wide strategy would improve the quality and consistency of engagement, which will in turn inform service planning and delivery.

A stakeholder engagement strategy would integrate current initiatives, such as the two major networks that consult with health planning staff and clinicians. But it will also need to extend the feedback gathered from families, carers and volunteers, and from the peak bodies that represent them. 

Appendix one - Response from the Agency

Appendix two -  National and NSW data on palliative care activity and outcomes

Appendix three -  About the audit

• ageing ICT equipment and inadequate wireless networks
• variable student access to devices at school
• variable teacher access to centrally provided devices for use outside of the classroom.

Information and communication technologies (ICT) are pervasive in modern life. Australian research has identified that the workforce demand for digital literacy and advanced digital skills is growing across most areas of work. There is broad agreement internationally and in Australian school systems that digital literacy is a core skill for the workforce and students will need to be confident with ICT. Education systems around the world are using ICT in classrooms to support learning and employment goals.  

The New South Wales Department of Education’s (the Department’s) overall strategic directions for teaching and learning with ICT are set in the 'Strategic Information Technology Plan 2016–19'. The Department centrally provides a base level of resources to schools for ICT and schools supplement funding from their existing school budget and Parents and Citizens Associations. Each school decides how to allocate these funds to meet local needs. Schools also set expectations for how teachers and students will use technology to help deliver outcomes.

This audit assessed how well New South Wales public schools are using ICT to improve teaching and learning. It focussed on planning and teacher and student use of ICT. We examined whether:

• the Department identifies key strategic opportunities to enhance the use of ICT platforms and technologies in schools
• teachers are integrating ICT into classroom practice
• the Department monitors the impact of ICT on student learning.

Old equipment and wireless networks are not keeping pace with modern demands

The Department’s vision for ICT is to enable ‘any learning opportunity, anywhere, anytime’. This vision is at risk due to an ageing stock of devices and wireless networks. The average age of devices in New South Wales schools is over four years. Older devices are less reliable, require greater maintenance and support, and cannot run demanding applications. Further, many school wireless networks are beyond the end of their useful life. This limits the number of teachers and students who can access online content on wireless networks at the same time.

The central funding model for ICT in schools is not meeting current needs

Funding for the Technology for Learning program to deliver ICT in schools has not increased since 2004, despite an increase in the number of students and emphasis placed on ICT in teaching and learning during this time. Schools supplement funding for ICT from their existing school budget and Parents and Citizens Associations.  

The Department’s current funding model for ICT is not adequately addressing a growing gap in the provision of contemporary ICT in classrooms between schools able to access funding from other sources and those which cannot. The Department needs to review whether the Technology for Learning program is equitable in equipping all schools with the modern technology needed to achieve its vision.

Many teachers are not provided with devices for use outside of the classroom

School Principals we interviewed reported that technology is an essential part of a modern classroom and teacher access to devices outside of the classroom can impact how they use ICT. This is because, in practice, teacher working days extend outside classroom hours and teachers need access to devices for activities such as lesson preparation and student assessment. The Department provides teachers with access to a suite of software tools for these tasks.

The Commonwealth Government’s Digital Education Revolution program provided teachers of secondary school students with laptops from 2009 to 2013. The Department’s evaluation of the Digital Education Revolution program found that teachers reported greater confidence with, and use of, ICT throughout the program.  
Providing desktop computers, laptops or tablets for teachers is now a school level decision and arrangements vary across schools. Each school must trade-off between allocating devices for students and teachers. Most other States and Territories provide all teachers with a laptop for use in and outside of the classroom or offer subsidised access to one.  

There is limited teacher professional learning in the use of ICT

The Department’s research has identified that professional learning is an important factor in how effectively teachers use ICT to enhance teaching and learning. Despite this, the Department provides few courses on using ICT in the classroom directly, and most of these are offered in Sydney. This limits accessibility for teachers outside of the metropolitan area. Schools we visited reported that the costs of courses and providing relief teachers limits the number of external courses or events that teachers attend, especially for rural and regional schools. Increasing the use of online learning would improve access for teachers in these areas.  

The Department is not adequately monitoring trends in professional learning in ICT or evaluating the overall effectiveness of courses. A recent upgrade to the professional learning system may provide the Department with better quality data to do this.

Greater monitoring and reporting on technology use in schools is required

The Local Schools, Local Decisions policy gives schools greater authority to make strategic decisions on the use of ICT appropriate to their local contexts. To support this, the Department needs to better monitor current trends, and identify emerging needs to determine future direction and how best to support schools.  

For example, the Department does not currently know how many devices are allocated to teachers or how many schools have implemented a student Bring Your Own Device scheme. This affects how schools are using ICT, and places demand on the network and the type of support the Department must provide. An assessment of the ICT maturity of schools would help the Department target its resources to schools requiring greater assistance with planning.

The Department does not regularly monitor or report on student capabilities with ICT. A national assessment found that the ICT literacy of a sample of Year 6 and Year 10 New South Wales students fell between 2011 and 2014. The fall was greater in New South Wales than in other States and Territories. Without more regular assessment or reporting, the reasons behind this fall and the distribution of student capabilities between schools will remain unknown. 

By July 2018, the Department of Education should:

1. Review the Technology for Learning program and school ICT support resourcing to determine whether resourcing is adequate for modern school requirements.
    
2. Develop a program to improve wireless networks in all NSW schools, for instance by expanding the Connecting Country Schools Program to all NSW schools.  
    
3. Implement an assessment of school ‘ICT maturity’ and use this to target assistance to those schools requiring support with forward planning for ICT.
    
4. Improve the use of evidence to inform plans and strategies, including:
   • more detailed monitoring of teacher and student access to and use of ICT
   • evaluating the impact of teacher professional learning on student outcomes 
   • further examining the links between ICT and student outcomes.
      
5. Improve teacher access to devices for use outside of the classroom to improve how effectively they integrate ICT into teaching and learning.
    
6. Improve teacher professional learning by providing more:
   • online learning opportunities for teachers in regional and remote areas
   • courses focused on pedagogy to make best use of ICT.
      
7. Identify the ICT skills students need, and provide teaching resources to develop these skills and monitor their achievement.

Appendix One - Response from the Agency

Appendix Two - About the audit

Appendix Three - Performance auditing

RMS’ SMC contract management includes most elements of good practice, including governance and dispute resolution mechanisms. However, key elements are missing which reduces its effectiveness.

Roads and Maritime Services (RMS) is responsible for the Sydney region State roads network This includes over 2,800 kilometres of roads and associated road corridor infrastructure such as bridges, tunnels and drainage structures. RMS divides the network into three geographical areas: south, west and north zones.

In 1995, RMS first outsourced road corridor infrastructure maintenance for the north zone through a Performance Specified Maintenance Contract (PSMC). The current 10-year PSMC for the north zone will expire in October 2018. Prior to November 2013, RMS maintained roads in the south and west zones through its Road and Fleet Services unit. 

In November 2013, RMS outsourced road maintenance services for the south and west zones using Stewardship Maintenance Contracts (SMC). The contracts run for seven years with an option for a further three years at RMS’ discretion. RMS estimated that the annual cost of these contracts was around $240 million in total. In March 2018, the contract prices are due to be reset by negotiation to reflect the contractors’ experience with, and better information about, the road networks and routine maintenance requirements. 

The SMC model adopted stewardship principles to improve value for money. RMS defined stewardship principles as a broad set of values, attitudes and behaviours, required of the contractor to effectively manage the assets on behalf of RMS. The SMC also includes commercial principles, such as linking risk to reward, and a performance framework where outcomes drive performance.

This audit assessed whether RMS had effectively managed the outsourcing of road maintenance in the Sydney region south and west zones. In making this assessment, we answered the following questions:

1. Did RMS justify the decision to adopt the SMC model?
2. Do SMCs include key performance indicators (KPIs) and incentives which promote efficiency and effectiveness? 
3. Does RMS collect high quality information on contractor performance and take action to correct performance deficiencies?
4. Are the expected benefits being achieved?

The Stewardship Maintenance Contract

RMS justified adopting the SMC model and included KPIs to drive efficiency and effectiveness

The SMC model includes features that RMS had not previously used for road maintenance contracts. These included adopting stewardship principles and transferring price risk to the contractor over time as the contractor becomes familiar with the assets being maintained.

The SMC model meets RMS’ requirements for flexibility in pricing models, the need for collaboration in asset maintenance planning, promoting innovation and effective performance management.

RMS used many good practices to develop the SMC model, including:

• preparing a robust business case comparing the SMC model to RMS maintaining the road network itself, as well as assessing whether two other contracting models
  (traditional and alliance) would meet its requirements
• assessing experiences with similar arrangements in other jurisdictions and identifying elements that worked to get the best outcomes
• developing a robust performance framework, which included a mix of efficiency and effectiveness KPIs that reflected NSW Government policy and RMS priorities
• incorporating risk and reward incentives delivered through cost sharing arrangements which change as the contract matures
• using a contract duration that supports RMS priorities and provides an incentive for better quality outcomes.

RMS uses data provided by the contractor to measure performance and calculate payments to the contractor. The SMC includes a specific sanction if RMS finds that the contractor provided incorrect performance data, but no specific sanction if the contractor provides incorrect financial data. If RMS finds that the contactor provided incorrect performance or financial data, RMS can only recover over-payments which may have been made using the incorrect data.  

To provide a stronger incentive for the contractor to ensure data it provides is accurate, RMS should consider whether to incorporate stronger sanctions when negotiating the commercial reset due in mid-2018 for south and west zones. RMS should also consider this for the new contract for the north zone when the current PSMC contract expires in October 2018.

RMS' contract management approach and benefits realization

RMS can improve the effectiveness of its oversight and management of the SMC

RMS does not have SMC specific contract procedures to guide its contract managers. Consequently, RMS has not exercised several significant SMC requirements, such as having the contractors account for an efficiency dividend in their pricing at the start of each three-year works period. Effective contract management should be supported by contract specific procedures, with explanations of, and allocation of responsibility for, the various interventions that RMS may be required to exercise in the SMC.

Performance and financial reporting under the SMC is based on a mix of RMS and contractor provided data. While there are a range of audits of contractor provided performance and financial data that RMS can conduct each year under the SMC, it does not have a schedule of audits it will conduct and when.  
During the first year of the SMC, RMS commissioned some limited audits of financial data. In the first three years of the SMC, RMS did not conduct any audits of performance data. Had there been SMC specific procedures in place, this would have reduced the risk of RMS not implementing a systematic audit program to give it reasonable assurance on the quality of the data that the contractor has provided. This is important because the data is used to measure performance and calculate contractor payments.

RMS has been aware of data quality issues since 2015. While RMS advised that it commenced addressing some data quality issues in response to a series of reviews conducted in 2015, a recent internal audit report indicates that RMS has not resolved the data quality issues.  

RMS achieved benefits in the first year, but has not tracked benefits since

As part of the business case, RMS agreed to implement a benefits realisation strategy, including a benefits tracking tool. RMS commenced tracking benefits, but did not establish a comparative baseline pre-SMC on non-financial benefits, and has not tracked benefits past year one.

In 2015, a benchmarking study commissioned by RMS found that it had achieved 80 per cent of the expected recurrent cost savings and other benefits, such as improved workplace safety, in the first full year of the SMC. However, there was no clear baseline to measure
non-financial performance. The study was qualified due to gaps in available data. The study also did not reconcile the actual one-off transition costs to the business case estimate.

During the course of the audit, RMS advised that it intends to repeat this type of study to determine whether it has achieved all expected benefits (and their value), and that it would use the results to inform its negotiation with the SMC contractors as part of the commercial reset due in mid-2018.

Roads and Maritime Services is responsible for the State Roads network in the Sydney region

Roads and Maritime Services (RMS) is responsible for the Sydney region State roads network. This includes over 2,800 kilometres of roads and associated road corridor infrastructure such as bridges, tunnels and drainage structures. The network is divided into three geographical areas: south, west and north zones. Prior to November 2013, RMS maintained roads in the Sydney region south and west zones through its Road and Fleet Services unit.  

In 1995, RMS first outsourced road corridor infrastructure maintenance for the north zone through a Performance Specified Maintenance Contract (PSMC). The current 10-year PSMC for the north zone will expire in October 2018. This contract is worth around $35 million per annum.  

NSW Government priorities and road maintenance

Efficient and effective road maintenance contributes to the following NSW Government priorities:

• improving road travel reliability
• ensuring on-time running of public transport
• reducing road fatalities
• improving government services
• keeping our environment clean.

The NSW Commission of Audit recommended outsourcing the maintenance of State roads

The NSW Commission of Audit in its Final Report on Government Expenditure (May 2012) recommended contestability as an appropriate strategy to consider for improving road maintenance service delivery for State roads.  

The Commission benchmarked RMS’ road surface quality and cost per lane kilometre against those of Western Australia, Victoria, and Queensland. This showed that New South Wales lagged the other states on both these measures.  

The Commission noted that RMS had conducted two independent reviews to examine the potential for extending road maintenance contestability. The Commission found that there was inadequate and inconclusive benchmarking to establish the efficiency of RMS’ Road and Fleet Services unit when compared to outsourcing. It recommended that RMS bring forward a proposal to conduct a competitive tender for the road maintenance of the Sydney region south zone road network to inform the feasibility of a progressive rollout of road maintenance contestability across other areas of the State. In August 2012, the NSW Government adopted the Commission’s recommendation.

The NSW Government introduced road maintenance contestability through Stewardship Maintenance Contracts

In April 2013, the NSW Government announced that it would introduce road maintenance contestability across the Sydney region, using a Stewardship Maintenance Contract (SMC) model to improve value for money. In doing so, it excluded RMS’ Road and Fleet Services unit from tendering.  

The SMC model is based on the following key commercial and performance principles set by RMS:

• performance driven by outcomes
• flexible and adaptable
• transparent and measurable
• linking risk to reward
• continuous improvement
• criteria for selection of, and transition to, different payment models.

The following key stewardship principles underpin the SMC’s broad set of values, attitudes and behaviours, which are required of the contractor to effectively manage the assets on behalf of RMS:

• putting RMS’ customers (road users and the general public) first and being responsive to them
• being responsible and accountable for the outcomes resulting from the management of the assets
• managing the assets diligently, efficiently and effectively with limited direction from RMS
• working collaboratively with RMS to deliver services that are tailored to meet RMS’ evolving needs
• acting with integrity and transparency in performing the services
• performing the services in the best interests of RMS and asset users.

Other key features of the SMC include:

• service requirements which describe the scope of the services, and the standards the contractor must meet
• a commercial framework which defines how payments are structured, how performance assessment will impact on payments and outlines the key commercial principles. SMCs primarily divide payments into two main mechanisms, these being the priced component (or fixed price) and the target cost calculated as follows:
  • fixed price – the contractor is paid a pre-agreed amount for specific services being provided, regardless of the actual costs incurred
  • target cost – RMS and the contractor agree on a target cost for a project, and any cost overruns or underruns are shared between them
• a performance framework which provides mechanisms for assessing contractor performance. This includes a comprehensive listing of the key result areas (KRAs) and key performance indicators (KPIs) against which RMS measures the contractor’s performance. The framework also outlines the scoring methodology that RMS uses to determine whether the contractor’s bid margin (profit and overheads) is reduced due to less than satisfactory performance or whether a bonus is paid if a threshold performance score is exceeded.

Road maintenance under SMCs for Sydney region south and west zones commenced in November 2013

In November 2013, RMS awarded SMCs to the Leighton Boral Amey consortium, now named Ventia Boral Amey (VBA), for the south zone and the DownerMouchel (DM) consortium for the west zone. The contracts run for seven years with an option for a further three years at RMS’ discretion. In April 2014, full services commenced following a four-month transition period. RMS estimated that the annual cost of these contracts was around $240 million in total. In March 2018, the contract prices are due to be reset by negotiation to reflect the contractors’ experience with, and better information about, the road networks and routine maintenance requirements. 

1. Roads and Maritime Services should consider whether to incorporate stronger sanctions in the Stewardship Maintenance Contract if the contractor provides incorrect performance or financial data to RMS, when:
    
   1. negotiating the commercial reset for the next works period with the Sydney region south and west zone contractors due in July 2018.
   2. finalising a new SMC contract for the Sydney region north zone, due to commence in October 2018.

Roads and Maritime Services should, by September 2017:

2.  Review its contract management framework for SMCs to ensure that all authorities and accountabilities of
     contract managers are clearly defined, including:

a) accountability and procedures for exercising all operational clauses in the SMC where RMS may opt to, or be required to intervene, or make a decision

b) authority to approve or initiate the interventions RMS is required to, or may, exercise under the SMC

c) the audits that RMS will conduct to systematically validate the performance and financial data that the SMC contractors provide

d) the accountabilities of RMS contract managers to systematically review audits and quality reviews that the SMC contractors must conduct to demonstrate compliance with their service plans

e) the accountabilities of RMS contract managers to check that the monthly and annual reports provided by SMC contractors do not contain errors, omissions or inaccuracies.

3.  Improve its management of benefits realisation by:

a) initiating a further benefits realisation review and record the benefits delivered against those
    estimated following the tender process, including the one-off transition costs

b) identify any benefits, including savings, not yet attained and develop strategies to address any short-falls

c) establish a tool to track the ongoing realisation of benefits.

Appendix One - Response from Roads and Maritime Services

Appendix Two - Better practice contract management framework

Appendix Three - About the audit

NorthConnex is a nine-kilometre tolled motorway tunnel between the M1 Pacific motorway at Wahroonga and the M2 Hills motorway at West Pennant Hills. The total cost for the project is $3.1 billion. NorthConnex will be funded through toll charges, and contributions from the NSW and Australian Governments of up to $405 million each. In January 2015, the NSW Roads Minister signed the final contracts for NorthConnex.

By December 2017, the Department of Premier and Cabinet should:

1. publish an updated ‘Unsolicited Proposals – Guide for Submission and Assessment’ which clarifies obligations with requirements in other NSW Government policies such as the NSW PPP guideline and Infrastructure Investor Assurance Framework. The update should require:

a) a business case to be prepared, and a business case gateway review completed, as part of the assessment of the detailed proposal (currently stage 2)

b) probity reports must be completed and considered before the decision to proceed to the next stage.
 

The Department of Premier and Cabinet and NSW Treasury should immediately:

2. improve record keeping to ensure compliance with the State Records Act 1998 and the NSW Government Standard on Records Management.

Appendix One - Response from Agency

Appendix Two - Governance Arrangements for NorthConnex

Appendix Three - About the Audit

• a high-value piece of equipment primarily used for diagnosing cancer - Positron Emission Tomography and Computed Tomography (PET-CT) scanners
• a small sample of lower value but critical medical equipment known as biomedical equipment.

Medical equipment needs to be properly managed over its lifecycle, from planning to acquisition, operation and disposal, to ensure patient safety and quality of care.

This audit assessed how well NSW hospitals managed medical equipment to meet the needs of patients. We examined the management of:

1. Positron Emission Tomography and Computed Tomography (PET-CT) scanners, a high-value piece of equipment commonly used for diagnosing cancer
2. a small sample of lower value but critical medical equipment known as biomedical equipment.  

We examined five hospitals for this audit: Lismore Base Hospital (in the Northern NSW Local Health District (LHD)), Liverpool Hospital (South Western Sydney LHD), Nepean Hospital (Nepean Blue Mountains LHD), Royal Prince Alfred Hospital (Sydney LHD) and Westmead Hospital (Western Sydney LHD).

Management of PET-CT scanners

PET-CT scanners were well utilised and reports were promptly sent to referring doctors

PET-CT scanners in Liverpool, Westmead and Royal Prince Alfred Hospitals were utilised to over 85 per cent of capacity. Utilisation at Nepean Hospital (around 60 per cent) was lower due to the age of the equipment and insufficient ‘uptake rooms’ for patients to receive radioactive injections. Lismore Base Hospital had a lower population to service and scheduled its PET-CT patients into three days a week to optimise efficiency.

PET-CT services were generally available to patients in a timely way and reports were promptly sent back to referring doctors. While clinicians we interviewed advised that there was generally no delay in patients accessing PET-CT scanners, only one hospital collected patient waiting time data to confirm this view.

Funding of PET-CT scans is complex

The funding of health services in NSW public hospitals involves a complex arrangement between the Australian and NSW Governments. In 2015–16, 10 per cent of PET-CT scans were inpatient services (funded mostly by NSW Health), 60 per cent were Medicare-funded outpatient services, and the remaining 30 per cent were privately referred outpatient services not funded by Medicare. Service costs for privately referred scans not funded by Medicare were met by a range of sources, including hospitals’ general purpose funds and patient out-of-pocket charges. Across the five hospitals, out-of-pocket charges varied and ranged from $250 to $950 per scan.

Better performance reporting could enable better planning of PET-CT scanners

NSW Health has delegated the planning functions for many pieces of high-value medical equipment, including PET-CT scanners, to Local Health Districts. This is intended to ensure local decision-making that is responsive to local community needs.

While local planning and service delivery is delegated to each Local Health District, under the Health Administration Act 1982, the Secretary of NSW Health is responsible for planning the provision of comprehensive, balanced and co-ordinated health services throughout New South Wales.

NSW Health could enable better service delivery and planning by collating and sharing performance information about PET-CT services across Local Health Districts.  

Equipment replacement planning was unclear 

Planning for future replacement of PET-CT scanners at the hospitals we examined was unclear, including when equipment would be replaced and what funding model might be applied. A better practice would be to have a clear equipment replacement plan for existing scanners that would ensure clarity about when equipment will be replaced, whether the replacement scanner should be leased, purchased or shared, and possible funding sources.

Management of biomedical equipment 

Equipment testing and maintenance did not always comply with intervals set under the Australian/New Zealand Standard All hospitals we examined adopted the Australian/New Zealand Standard 

All hospitals we examined adopted the Australian/New Zealand Standard 3551 ‘Management programs for medical equipment’ (the Standard) for managing medical equipment, the purpose of which is to ensure that equipment is safe and suitable for use. The Standard requires the regular testing and maintenance of biomedical equipment at predetermined intervals.  

Our review of three years of service records for 50 items of biomedical equipment found that:

• nineteen (38 per cent) items of equipment were tested and maintained within the intervals determined by hospitals under the Standard
• five (ten per cent) had at least one instance where they were tested and maintained less than 30 days later than when the work was due
• thirteen (26 per cent) had at least one instance where they were tested and maintained one to six months later than when the work was due
• six (12 per cent) had at least one instance where they were tested and maintained more than six months later than when the work was due
• seven (14 per cent) were lost, removed from clinical use or unable to be unidentified.

The Standard envisages that there may be circumstances when testing and maintenance does not occur according to schedule, and sets out a procedure that should be followed when testing and maintenance is overdue. This procedure was not followed in any of the hospitals we reviewed.  

Two out of five audited hospitals used risk rating to oversee equipment maintenance

Only two out of five hospitals we examined used risk rating, under which equipment is classified according to clinical risk, to prioritise equipment maintenance and to determine appropriate frequencies for equipment testing and maintenance.  

Some hospitals had inadequate oversight of work performed by external contractors

There was variable oversight of outsourced service contracts for high-risk biomedical equipment. In some cases, hospitals did not maintain complete histories of testing and maintenance work performed by contractors. Some contractors had incorrectly recorded items they had tested, or had refused to provide details of testing and maintenance performed.

New peer review process may improve assurance over testing and maintenance

NSW Health has started a peer review process in a small number of hospitals. This process covers a range of performance indicators relating to equipment management practices, including the auditing of test and maintenance records for two pieces of equipment per hospital. There is opportunity to build upon this effort by including all hospitals in the peer review process, and by expanding the sample of equipment subject to records audit.  

Hospitals’ record keeping of testing and maintenance service histories was inefficient and inadequate

The Standard requires that adequate and traceable equipment maintenance histories be kept. We found that hospitals’ record keeping of equipment service histories was inefficient and inadequate. None of the hospitals used an information system that provided the full three-levels of capability outlined below:

• storing equipment information electronically, allowing easy retrieval
• managing service requests and holding full service histories and test results
• automatically generating reports to allow risk based prioritisation of equipment maintenance, repairs and replacements.

There is an urgent need to implement the state-wide asset management system for biomedical equipment

Hospitals advised that the current outdated systems will be replaced by a state-wide asset management system, Asset and Facilities Management Online, though this implementation has experienced delays.

There was good governance over equipment acquisition, replacement and disposal

All hospitals had formal processes for acquiring and replacing biomedical equipment, including management committees to oversee equipment needs. Equipment disposal processes were aligned with relevant standards and policies.

All hospitals purchased the majority of their biomedical equipment through HealthShare, the central procurement agency of NSW Health. This contributed to cost savings across the health system.

Management of medical equipment in the NSW public health system

In New South Wales, responsibility for the management of public hospitals is devolved from the NSW Ministry of Health to 15 Local Health Districts and two Speciality Health Networks.The Secretary of NSW Health retains a function under the Health Administration Act 1982 to plan the provision of comprehensive, balanced and co-ordinated health services throughout the State.

Every year, the Ministry of Health and Local Health Districts sign a service agreement that sets out the expected performance from Local Health Districts and the funding they will receive to provide their services. Under these arrangements, responsibility for managing medical equipment is delegated to Local Health Districts.  

Medical equipment is used to diagnose, treat and manage patients. It includes items as diverse as patient beds, dialysis machines, operating tables and heart monitors. The good management of medical equipment contributes to ensuring patient care and safety, as well as keeping the cost burden on the public health system low.

The New South Wales public health system uses a wide range of medical equipment. Most of this equipment is used in hospital settings, however, some is also used in community health centres and patients’ homes. The cost of individual items ranges from less than $100 to several million dollars. In total, about $1.2 billion, or six per cent of NSW Health’s total asset value, was for medical equipment.

The approach used to manage medical equipment varies between hospitals, and between expensive and less-expensive items. Different service models are also used, for example, some items may be purchased in one hospital, but leased in another.  

About the audit

This audit assessed how well NSW public hospitals managed medical equipment to meet the needs of patients. We looked at the lifecycle of biomedical equipment, which comprises planning, acquisition, operation and maintenance, and then replacement and disposal. The audit questions in relation to each stage of the lifecycle are summarised in Exhibit 1. 

By June 2018 

1. NSW Health should review all services provided by Local Health Districts which use high-value medical equipment (with establishment cost that exceeds $3 million), to determine whether state-level coordination, service benchmarking and equipment usage reporting is warranted.

2. NSW public hospitals offering PET-CT services should collect and use patient waiting time data (the difference between the date of referral and the actual date of the scan) as part of improving service efficiency and meeting patient needs.

3. Local Health Districts should ensure that there is a formal equipment replacement plan at the time of procuring high-value equipment, for both new and existing services. The plan should include an estimated time of replacement. The Ministry of Health should regularly review capital funding implications from these planned equipment replacements.

By June 2019

4. NSW public hospitals should review internal business rules and processes for biomedical equipment management to ensure that:  

a) equipment is accessible by service technicians for testing and maintenance work, including establishing internal processes to assist service technicians in gaining access to equipment that has missed previous testing and maintenance attempts in accordance with the Australian/New Zealand Standard 3551

b) adequate maintenance records are kept, including descriptions of testing and maintenance work carried out in accordance with the Australian/New Zealand Standard 3551

c) there is regular reporting to Local Health District Chief Executives on the compliance of equipment testing and maintenance, including equipment that is tested or maintained later than scheduled intervals

d) there is specified statement of risk tolerance for late equipment testing and maintenance and mechanisms to appropriately prioritise equipment testing and maintenance.

5. Ministry of Health should encourage that all NSW public hospitals have their biomedical equipment management practices reviewed under the new peer review process, and that the review sample from each hospital be increased to more than two pieces of equipment per hospital.

6. Ministry of Health should complete the implementation of AFM Online for biomedical equipment management.

Appendix One - Response from NSW Health

Appendix Two - About the Audit

NSW Government agencies use contingent labour to help deliver services to the community. The NSW Public Service Commission (PSC) defines contingent labour as people employed by a recruitment agency and hired by government agencies to provide labour or services. Agencies use contingent labour to fill a gap in skills or capability, for example, to fill a position while a staff member is on leave or where specialist knowledge may be needed on a short-term basis. The PSC estimated that in 2016 the contingent workforce represented 2.3 per cent of the public sector workforce, equivalent to 7,571 full-time employees.

The PSC recommends that contingent labour only be used when it is the most efficient and effective option available to respond to an agency’s business needs. It also recommends that agencies’ use of contingent labour be informed by workforce planning. 

Government spending on contingent labour has increased significantly over the last five years, from $503 million in 2011–12 to $1.1 billion in 2015–16. To reduce spend in this area, the NSW Government has introduced the Contingent Workforce Renewal Strategy, overseen by NSW Procurement. The Strategy aims to achieve greater efficiency and effectiveness in the use of contingent labour. It has four pillars:

• prequalification scheme – a list of approved contingent labour suppliers
• vendor management system – an information system to manage contingent workers
• managed service provider – a recruitment agency broker
• contractor management organisations – organisations that manage a contingent labour database, which agencies can source labour from.

The prequalification scheme is mandatory for public sector agencies. Agencies are progressively rolling out the other pillars. The vendor management system and managed service provider are together called ‘Contractor Central’.

Within the context of sector reform aimed at promoting efficiency and effectiveness, the objective of this audit is to assess whether agencies’ approach to purchasing and managing their contingent workforce meets business needs and delivers value for money. In making this assessment, we reviewed three agencies each at a different stage of the reform:

• Department of Education (Education) – Contractor Central introduced in August 2015
• Department of Industry (Industry) – Contractor Central introduced in November 2016, after our review
• Transport for NSW (Transport) – Contractor Central not in place.

The use of contingent labour neither informs nor is informed by agency level workforce plans

None of the three agencies we reviewed had an agency level workforce plan in place. Agencies could not demonstrate that they had analysed their use of contingent labour at an agency level, including how it is being used to address any skills gaps. An agency’s executive is responsible for ensuring that an agency level workforce plan is in place. An agency level workforce plan helps hiring managers to make decisions on the best resource strategy to meet their business needs. This is important because contingent labour should only be engaged after considering all other recruitment options and the agency’s workforce plan.

Contingent workforce data is not always reliable or accurate

The accuracy and reliability of contingent workforce data varied significantly across the three agencies we reviewed. In Industry and Transport, information on contingent labour is difficult to obtain because it must be drawn from different data sources, affecting its accuracy, reliability and timeliness. This information is also incomplete, with these agencies not having a full picture of their contingent workforce. Quality data is important because it improves an agency’s capacity to plan and monitor its use of contingent labour to ensure it meets business needs.

At the time of our review, only Education, through Contractor Central, was able to obtain timely and accurate data on its use of contingent labour. Contractor Central has also improved its reporting capability, with the agency’s executive now receiving quarterly reports on its contingent workforce. In contrast, executives in Industry and Transport received ad-hoc reports on the use of contingent labour that only gave them limited oversight of their contingent workforce.

Long tenure of contingent workers is an issue in agencies

We found that the maximum tenure of contingent labour varied across agencies from nine to more than 20 years. In Education and Transport, staff reported that hiring managers assume contingent workers are automatically renewed at the end of their contract, with no formal consideration about whether contingent labour is still needed. Also, contingent labour is used for significant capital projects in the information technology and infrastructure areas where a project may run for several years.

None of the agencies reviewed undertook an analysis to determine how to reduce tenure while ensuring business needs are met. This is particularly important for long-term use of contingent labour for large capital projects. Understanding whether contingent labour represents best value compared to other recruitment options, such as secondments or temporary employment, is essential. Contingent workers are engaged under different working conditions to employees. Long tenure can pose an industrial relations risk to agencies because contingent workers may believe they are entitled to the same working conditions as employees.

On and off-boarding processes could be strengthened

Agencies have processes to engage and release contingent labour, also called on boarding and off-boarding. This includes access to IT systems, building access, and the return of property. However, not all agencies had on boarding or off-boarding checklists with specific requirements for engaging or releasing contingent labour. In addition, agencies’ off boarding guidelines did not always provide for knowledge transfer. This was identified as a key risk by staff because it is important to ensure that critical skills and knowledge are retained.

Risk that agencies are being overcharged when engaging contingent labour

We found that in agencies without Contractor Central, there is limited assurance that recruitment agencies charge in line with the prequalification scheme fees. NSW Procurement estimates that the government was overcharged $1.3 million in 2015–16. In addition, there is a risk that hiring managers do not have sufficient information to benchmark pay rates when negotiating contingent labour engagements. Agencies with Contractor Central may be more likely to get reasonable rates by using a recruitment broker who has specialised market knowledge.

No system in place to monitor the performance of contingent workers

None of the agencies we reviewed had a system in place to monitor the performance of their contingent workforce at an agency level to ensure it delivers value for money. Hiring managers are not required to evaluate whether contingent labour delivers the services for which they are hired. For example, hiring managers do not routinely assess and centrally document the quality of services provided, including whether services are delivered on time and within budget. This means contingent workers who are not performing may be re-hired by other managers or agencies. With the implementation of Contractor Central, there is the means to capture agency-wide information on the performance of contingent workers.

Contractor Central has the potential to improve value for money

Contractor Central has the potential to improve value for money. This is because the recruitment broker has specialised market knowledge and is able to promote competition, and benchmark and negotiate pay rates. In addition, the new software can streamline invoice processing and ensure correct supplier rates are charged. Education reports that it achieved a net saving of $944,600 from August 2015 to May 2016 due to the introduction of Contractor Central. Industry also expects to achieve similar results with Contractor Central, which it advised was implemented in November 2016.

The Department of Industry and Transport for NSW should, by December 2017:

1. improve the accuracy and reliability of their data on contingent labour

2. routinely report the use of contingent labour to agency executive.

The Department of Industry, Department of Education, and Transport for NSW should:

by December 2017:

3. ensure agency-wide on-boarding and off-boarding guidelines or checklists detail the specific requirements for engaging or releasing contingent labour, including provisions for knowledge transfer.

by March 2018

4. ensure that contingent labour informs and is informed by workforce planning, by:

• analysing agency-wide business needs, staff capability, and skills gaps
• understanding how gaps are filled by contingent workers or other recruitment options
• assessing whether long-term contingent worker engagements are the most economical and effective labour option
• evaluating whether contingent workers meet agency business needs and deliver value for money.

5. assess and centrally document the performance of their contingent workforce to ensure that services are delivered as contracted

6. implement processes to ensure that hiring managers consider other recruitment options prior to engaging or re-engaging contingent workers.
    

Sector-wide learnings

This audit identified learnings that government agencies across the sector should consider when procuring and managing contingent labour:

1. Contingent workforce planning should be part of an agency’s broader workforce planning.

2. Using information systems to manage and procure contingent labour improves the accuracy, reliability and timeliness of contingent labour data. This information enables agencies to consistently assess contingent labour rates and to identify persistent skills gaps in their workforce.

3. Routine reporting of contingent labour to agency executives provides oversight of an agency’s use of contingent labour.

4. Hiring managers should consider all recruitment options, with advice from human resources staff, before engaging contingent labour to ensure that it is the most appropriate solution for a specific need.

5. Regularly assessing long tenure contingent labour engagements helps to ensure that such engagements are still the most economical and effective labour option.

6. Planning the engagement of contingent workers, including provisions for knowledge transfer, maximises the potential to obtain value for money from the use of contingent labour.

7. Assessing and centrally documenting the performance of contingent labour against agreed deliverables helps to ensure services are delivered as planned, including in terms of quality, and timeliness.

Appendix one - Response from the Department

Appendix two - Survey of disability service providers

Appendix three - About the audit