Reports
Matching skills training with market needs
In 2012, governments across Australia entered into the National Partnership Agreement on Skills Reform. Under the National Partnership Agreement, the Australian Government provided incentive payments to States and Territories to move towards a more contestable Vocational Education and Training (VET) market. The aim of the National Partnership Agreement was to foster a more accessible, transparent, efficient and high quality training sector that is responsive to the needs of students and industry.
The New South Wales Government introduced the Smart and Skilled program in response to the National Partnership Agreement. Through Smart and Skilled, students can choose a vocational course from a list of approved qualifications and training providers. Students pay the same fee for their chosen qualification regardless of the selected training provider and the government covers the gap between the student fee and the fixed price of the qualification through a subsidy paid to their training provider.
Smart and Skilled commenced in January 2015, with the then Department of Education and Communities having primary responsibility for its implementation. Since July 2015, the NSW Department of Industry (the Department) has been responsible for VET in New South Wales and the implementation of Smart and Skilled.
The NSW Skills Board, comprising nine part-time members appointed by the Minister for Skills, provides independent strategic advice on VET reform and funding. In line with most other States and Territories, the Department maintains a 'Skills List' which contains government subsidised qualifications to address identified priority skill needs in New South Wales.
This audit assessed the effectiveness of the Department in identifying, prioritising, and aligning course subsidies to the skill needs of NSW. To do this we examined whether:
- the Department effectively identifies and prioritises present and future skill needs
- Smart and Skilled funding is aligned with the priority skill areas
- skill needs and available VET courses are effectively communicated to potential participants and training providers.
Smart and Skilled is a relatively new and complex program, and is being delivered in the context of significant reform to VET nationally and in New South Wales. A large scale government funded contestable market was not present in the VET sector in New South Wales before the introduction of Smart and Skilled. This audit's findings should be considered in that context.
The Department needs to better use the data it has, and collect additional data, to support its analysis of priority skill needs in New South Wales, and direct funding accordingly.
- funding scholarships and support for disadvantaged students
- funding training in regional and remote areas
- providing additional support to deliver some qualifications that the market is not providing.
The Department needs to evaluate these funding strategies to ensure they are achieving their goals. It should also explore why training providers are not delivering some priority qualifications through Smart and Skilled.
Training providers compete for funding allocations based on their capacity to deliver. The Department successfully manages the budget by capping funding allocated to each Smart and Skilled training provider. However, training providers have only one year of funding certainty at present. Training providers that are performing well are not rewarded with greater certainty.
The Department needs to improve its communication with prospective students to ensure they can make informed decisions in the VET market.
The Department also needs to communicate more transparently to training providers about its funding allocations and decisions about changes to the NSW Skills List.
The Department relies on stakeholder proposals to update the NSW Skills List. Stakeholders include industry, training providers and government departments. These stakeholders, particularly industry, are likely to be aware of skill needs, and have a strong incentive to propose qualifications that address these needs. The Department’s process of collecting stakeholder proposals helps to ensure that it can identify qualifications needed to address material skill needs.
It is also important that the Department ensures the NSW Skills List only includes priority qualifications that need to be subsidised by government. The Department does not have robust processes in place to remove qualifications from the NSW Skills List. As a result, there is a risk that the list may include lower priority skill areas. Since the NSW Skills List was first created, new additions to the list have outnumbered those removed by five to one.
The Department does not always validate information gathered from stakeholder proposals, even when it has data to do so. Further, its decision making about what to include on, or delete from, the NSW Skills List is not transparent because the rationale for decisions is not adequately documented.
The Department is undertaking projects to better use data to support its decisions about what should be on the NSW Skills List. Some of these projects should deliver useful data soon, but some can only provide useful information when sufficient trend data is available.
Recommendation
The Department should:
- by June 2019, increase transparency of decisions about proposed changes to the NSW Skills List and improve record-keeping of deliberations regarding these changes
- by December 2019, use data more effectively and consistently to ensure that the NSW Skills List only includes high priority qualifications
Only qualifications on the NSW Skills List are eligible for subsidies under Smart and Skilled. As the Department does not have a robust process for removing low priority qualifications from the NSW Skills list, some low priority qualifications may be subsidised.
The Department allocates the Smart and Skilled budget through contracts with Smart and Skilled training providers. Training providers that meet contractual obligations and perform well in terms of enrolments and completion rates are rewarded with renewed contracts and more funding for increased enrolments, but these decisions are not based on student outcomes. The Department reduces or removes funding from training providers that do not meet quality standards, breach contract conditions or that are unable to spend their allocated funding effectively. Contracts are for only one year, offering training providers little funding certainty.
Smart and Skilled provides additional funding for scholarships and for training providers in locations where the cost of delivery is high or to those that cater to students with disabilities. The Department has not yet evaluated whether this additional funding is achieving its intended outcomes.
Eight per cent of the qualifications that have been on the NSW Skills List since 2015 are not delivered under Smart and Skilled anywhere in New South Wales. A further 14 per cent of the qualifications that are offered by training providers have had no student commencements. The Department is yet to identify the reasons that these high priority qualifications are either not offered or not taken up by students.
Recommendation
The Department should:
- by June 2019, investigate why training providers do not offer, and prospective students do not enrol in, some Smart and Skilled subsidised qualifications
- by December 2019, evaluate the effectiveness of Smart and Skilled funding which supplements standard subsidies for qualifications on the NSW Skills List, to determine whether it is achieving its objectives
- by December 2019, provide longer term funding certainty to high performing training providers, while retaining incentives for them to continue to perform well.
In a contestable market, it is important for consumers to have sufficient information to make informed decisions. The Department does not provide some key information to prospective VET students to support their decisions, such as measures of provider quality and examples of employment and further education outcomes of students completing particular courses. Existing information is spread across numerous channels and is not presented in a user friendly manner. This is a potential barrier to participation in VET for those less engaged with the system or less ICT literate.
The Department conveys relevant information about the program to training providers through its websites and its regional offices. However, it could better communicate some specific information directly to individual Smart and Skilled training providers, such as reasons their proposals to include new qualifications on the NSW Skills List are accepted or rejected.
While the Department is implementing a communication strategy for VET in New South Wales, it does not have a specific communications strategy for Smart and Skilled which comprehensively identifies the needs of different stakeholders and how these can be addressed.
Recommendation
By December 2019, the Department should develop and implement a specific communications strategy for Smart and Skilled to:
- support prospective student engagement and informed decision making
- meet the information needs of training providers
Appendix one - Response from agency
Appendix two - About the audit
Appendix three - Performance auditing
Parliamentary reference - Report number #305 - released 26 July 2018
Performance audit insights: key findings from 2014-2018
A report released today by the Auditor-General for New South Wales, Margaret Crawford, presents key findings from four years of performance audits. The report findings are presented around six areas of government activity including planning for the future, meeting community expectations for key services, investment in infrastructure, managing natural resources, ensuring good governance and digital disruption.
In this report, we present common findings and lessons from the past four years of performance audits, and offer insights to the public sector on elements of effective performance. We have analysed the key findings and recommendations from 61 performance audits tabled in the NSW Parliament between July 2014 and June 2018, spanning varied areas of government activity. We will also use this report to help determine areas of unaddressed risk across all parts of government, and to shape our future audit priorities.
Governments play an important stewardship role. Their decisions need to consider intergenerational equity by ensuring that investment strategies are sustainable. Governments also need to consider the impact of their decisions on different parts of the community. We recognise that governments face challenges in delivering programs and services, targeting complex social issues with finite resources.
Governments are changing how they deliver services to respond to citizen needs and deliver greater value for money. In this section, we reflect on audits that looked at how government entities are planning their activities to meet the needs of the community into the future.
State and local government exist to provide services to citizens, and citizens are playing a greater role in defining what services they want or need. Expectations about consultation, ease of access, timeliness, and customisation of services are rising. Governments face challenges to continually improve the way they plan and deliver services to meet these expectations. Governments also need to provide quality services for a growing and ageing population whilst working within a constrained financial environment.
Over the past four years, our performance audits have assessed aspects of State and local government services, including education, health services, disability support, corrective services, and many others. In this section, we draw together common findings that government entities should reflect on when providing services to the community.
The NSW Government’s 2018–19 Budget forecasts an $87.2 billion infrastructure investment program over the next four years. Infrastructure investment of this size carries significant opportunities and risks. Competition for resources is high and maintaining the capability to manage and deliver projects effectively is challenging. Governments also need to plan effectively to ensure infrastructure built today will meet future needs.
Over the past four years, we have looked at some of the ways NSW Government agencies justify and prioritise projects for funding, work with contractors to deliver projects, and track and report on progress. In this section, we draw together common findings from our audits that government entities should consider when planning future infrastructure projects.
Governments face challenges in balancing the use of natural resources to meet diverse interests, while supporting a sustainable natural environment for the future. They need to supply communities with water, produce energy, protect natural habitats, and support farming, industry, and economic development.
Some of our recent audits have considered how government agencies are managing natural resources and protecting the environment for future generations. In this section, we have drawn together common findings across our audits that government entities should consider in managing the environment and natural resources.
A range of checks and balances is needed to support public confidence in government decision making. To maintain trust, government agencies should act transparently, and in accordance with relevant legislation and policy. This is particularly important as the public sector increasingly engages with external partners to deliver services and provide a more contestable environment.
Good governance arrangements should result in improved service delivery and more effective and efficient use of resources. Our audits have looked at many different elements of governance, including making sure the necessary processes and workplace cultures are in place to help government entities achieve their aims. In this section, we have drawn together various aspects of governance that government entities should consider.
The global increase in digital technology provides governments with opportunities to interact with citizens in more immediate and responsive ways than was previously possible. Data can be used in powerful ways such as predicting future demand for services, targeting interventions, responding to crises, and evaluating outcomes. Governments face challenges in doing this while maintaining secure digital environments that protect citizen interests, privacy, and autonomy.
Our audits have assessed some of the ways that government entities are incorporating digital change into their work. In this section, we draw together common themes that governments could consider in protecting their digital assets, or expanding their digital capabilities.
Council reporting on service delivery
New South Wales local government councils’ could do more to demonstrate how well they are delivering services in their reports to the public, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford. Many councils report activity, but do not report on outcomes in a way that would help their communities assess how well they are performing. Most councils also did not report on the cost of services, making it difficult for communities to see how efficiently they are being delivered. And councils are not consistently publishing targets to demonstrate what they are striving for.
I am pleased to present my first local government performance audit pursuant to section 421D of the Local Government Act 1993.
My new mandate supports the Parliament’s objectives to:
- strengthen governance and financial oversight in the local government sector
- improve financial management, fiscal responsibility and public accountability for how councils use citizens’ funds.
Performance audits aim to help councils improve their efficiency and effectiveness. They will also provide communities with independent information on the performance of their councils.
For this inaugural audit in the local government sector, I have chosen to examine how well councils report to their constituents about the services they provide.
In this way, the report will enable benchmarking and provide improvement guidance to all councils across New South Wales.
Specific recommendations to drive improved reporting are directed to the Office of Local Government, which is the regulator of councils in New South Wales.
Councils provide a range of services which have a direct impact on the amenity, safety and health of their communities. These services need to meet the needs and expectations of their communities, as well as relevant regulatory requirements set by state and federal governments. Councils have a high level of autonomy in decisions about how and to whom they provide services, so it is important that local communities have access to information about how well they are being delivered and meeting community needs. Ultimately councils should aim to ensure that reporting performance is subject to quality controls designed to provide independent assurance.
Councils report extensively on the things they have done, but minimally on the outcomes from that effort, efficiency and performance over time.
Councils could improve reporting on service delivery by more clearly relating the resources needed with the outputs produced, and by reporting against clear targets. This would enable communities to understand how efficiently services are being delivered and how well councils are tracking against their goals and priorities.
Across the sector, a greater focus is also needed on reporting performance over time so that communities can track changes in performance and councils can demonstrate whether they are on target to meet any agreed timeframes for service improvements.
The degree to which councils demonstrate good practice in reporting on service delivery varies greatly between councils. Metropolitan and regional town and city councils generally produce better quality reporting than rural councils. This variation indicates that, at least in the near-term, OLG's efforts in building capability in reporting would be best directed toward rural councils.
Recommendation
By mid-2018, OLG should:
- assist rural councils to develop their reporting capability.
The Framework which councils are required to use to report on service delivery, is intended to drive good practice in reporting. Despite this, the Framework is silent on a number of aspects of reporting that should be considered fundamental to transparent reporting on service delivery. It does not provide guidance on reporting efficiency or cost effectiveness in service delivery and provides limited guidance on how annual reports link with other plans produced as part of the Framework. OLG's review of the Framework, currently underway, needs to address these issues.
Recommendation
By mid-2018, OLG should:
- issue additional guidance on good practice in council reporting, with specific information on:
- reporting on performance against targets
- reporting on performance against outcome
- assessing and reporting on efficiency and cost effectiveness
- reporting performance over time
- clearer integration of all reports and plans that are required by the Framework, particularly the role of End of Term Reporting
- defining reporting terms to encourage consistency.
The Framework is silent on inclusion of efficiency or cost effectiveness indicators in reports
The guidelines produced by OLG in 2013 to assist councils to implement their Framework requirements advise that performance measures should be included in all plans. However, the Framework does not specifically state that efficiency or cost effectiveness indicators should be included as part of this process. This has been identified as a weakness in the 2012 performance audit report and the Local Government Reform Panel review of reporting by councils on service delivery.
The Framework and supporting documents provide limited guidance on reporting
Councils' annual reports provide a consolidated summary of their efforts and achievements in service delivery and financial management. However, OLG provides limited guidance on:
- good practice in reporting to the community
- how the annual report links with other plans and reports required by the Framework.
Further, the Framework includes both Annual and End of Term Reports. However, End of Term reports are published prior to council elections and are mainly a consolidation of annual reports produced during a council’s term. The relationship between Annual reports and End of Term reports is not clear.
OLG is reviewing the Framework and guidance
OLG commenced work on reviewing of the Framework in 2013 but this was deferred with work re‑starting in 2017. The revised guidelines and manual were expected to be released late in 2017.
OLG should build on the Framework to improve guidance on reporting on service delivery, including in annual reports
The Framework provides limited guidance on how best to report on service delivery, including in annual reports. It is silent on inclusion of efficiency or cost effectiveness indicators in reporting, which are fundamental aspects of performance reporting. Councils we consulted would welcome more guidance from OLG on these aspects of reporting.
Our consultation with councils highlighted that many council staff would welcome a set of reporting principles that provide guidance to councils, without being prescriptive. This would allow councils to tailor their approach to the individual characteristics, needs and priorities of their local communities.
Consolidating what councils are required to report to state agencies would reduce the reporting burden and enable councils to better report on performance. Comparative performance indicators are also needed to provide councils and the public with a clear understanding of councils' performance relative to each other.
Recommendations
By mid-2018, OLG should:
- commence work to consolidate the information reported by individual councils to NSW Government agencies as part of their compliance requirements.
- progress work on the development of a Performance Measurement Framework, and associated performance indicators, that can be used by councils and the NSW Government in sector-wide performance reporting.
Streamlining the reporting burden would help councils improve reporting
The NSW Government does not have a central view of all local government reporting, planning and compliance obligations. A 2016 draft IPART ‘Review of reporting and compliance burdens on Local Government’ noted that councils provide a wide range of services under 67 different Acts, administered by 27 different NSW Government agencies. Consolidating and coordinating reporting requirements would assist with better reporting over time and comparative reporting. It would also provide an opportunity for NSW Government agencies to reduce the reporting burden on councils by identifying and removing duplication.
Enabling rural councils to perform tailored surveys of their communities may be more beneficial than a state-wide survey in defining outcome indicators
Some councils use community satisfaction survey data to develop outcome indicators for reporting. The results from these are used by councils to set service delivery targets and report on outcomes. This helps to drive service delivery in line with community expectations. While some regional councils do conduct satisfaction surveys, surveys are mainly used by metropolitan councils which generally have the resources needed to run them.
OLG and the Department of Premier and Cabinet have explored the potential to conduct state-wide resident satisfaction surveys with a view to establishing measures to improve service delivery. This work has drawn from a similar approach adopted in Victoria. Our consultation with stakeholders in Victoria indicated that the state level survey is not sufficiently detailed or specific enough to be used as a tool in setting targets that respond to local circumstances, expectations and priorities. Our analysis of reports and consultation with stakeholders suggest that better use of resident survey data in rural and regional areas may support improvements in performance reporting in these areas. Rural councils may benefit more from tailored surveys of groups of councils with similar challenges, priorities and circumstances than from a standard state-wide survey. These could potentially be achieved through regional cooperation between groups of similar councils or regional groups.
Comparative reporting indicators are needed to enable councils to respond to service delivery priorities of their communities
The Local Government Reform Panel in 2012 identified the need for ‘more consistent data collection and benchmarking to enable councils and the public to gain a clear understanding of how a council is performing relative to their peers’.
OLG commenced work in 2012 to build a new performance measurement Framework for councils which aimed to move away from compliance reporting. This work was also strongly influenced by the approach used in Victoria that requires councils to report on a set of 79 indicators which are reported on the Victorian 'Know your council' website. OLG’s work did not fully progress at the time and several other local government representative bodies have since commenced work to establish performance measurement frameworks. OLG advised us it has recently recommenced its work on this project.
Our consultation identified some desire amongst councils to be able to compare their performance to support improvement in the delivery of services. We also identified a level of frustration that more progress has not been made toward establishment of a set of indicators that councils can use to measure performance and drive improvement in service delivery.
Several councils we spoke with were concerned that the current approaches to comparative reporting did not adequately acknowledge that councils need to tailor their service types, level and mix to the needs of their community. Comparative reporting approaches tend to focus on output measures such as number of applications processed, library loans annually and opening hours for sporting facilities, rather than outcome measures. These approaches risk unjustified and adverse interpretations of performance where councils have made a decision based on community consultation, local priorities and available resources. To mitigate this, it is important to
- adopt a partnership approach to the development of indicators
- ensure indicators measure performance, not just level of activity
- compare performance between councils that are similar in terms of size and location.
It may be more feasible, at least in the short term, for OLG to support small groups of like councils to develop indicators suited to their situation.
Based on our consultations, key lessons from implementing a sector-wide performance indicator framework in Victoria included the benefits of:
- consolidation of the various compliance data currently being reported by councils to provide an initial platform for comparative performance reporting
- adopting a partnership approach to development of common indicators with groups of like councils.
Appendix one - Response from agency
Appendix two - Service delivery categorisation
Appendix three - Reporting targets and performance over time
Appendix four - Performance auditing
Appendix five - About the audit
Parliamentary reference - Report number #296 - released 1 February 2018
Police Rostering
Overall, the NSW Police Force (the Force) is not meeting its target for responding to urgent calls for assistance. Today, one in four complaints is about customer service. Victims of crime report issues including difficulties in contacting an officer and frustration at having to wait for an officer to return to duty to find out about their case. The timeliness of investigations is also at risk when follow-up is delayed. Seventy per cent of commanders surveyed indicated that rosters did not support continuity of investigations or victims’ needs.
We are also concerned that the current roster pattern may not be in the best interest of the health and safety of officers. Sick and injury leave is higher in NSW than other states and the Force is yet to assess the extent to which rosters may contribute to this. The Force also needs to do more to protect its general duties officers from fatigue in line with contemporary practice for managing shiftwork.
Parliamentary reference - Report number #173 - released 5 December 2007
Improving efficiency of irrigation water use on farms
The Department of Primary Industries (DPI) has a crucial role in assisting the irrigation industry to respond to the conflicting demands placed upon the state’s water resources. The State Plan emphasises the importance of water to agriculture, and gives DPI responsibility to use its training programs to help farmers continue to increase water use efficiency. DPI needs to work closely with agencies with which it shares responsibilities in the State Plan to ensure that its water use efficiency activities contribute to the State Plan goals. It needs to work closely with Treasury to document the services it will require to do this, the resources which will be needed, and how its success will be measured. This will ensure that DPI is well placed to continue to assist the NSW farming industry to adapt and survive periods of reduced water availability.
Parliamentary reference - Report number #172 - released 21 November 2007
Signal failures on the metropolitan rail network
Between 2004 and 2006, the number of signalling failures, signalling downtime and the number of trains delayed as a result of signal failures all fell. RailCorp’s on-time running performance improved over the same period. The fall in failures is a clear indication of improved performance. Changes in the definition of on-time and to the timetable during 2005 and 2006 however make it difficult to determine whether improvements in response downtime and signalling delays are due to a true performance improvement. To build upon this strong base, RailCorp needs to determine with more confidence the number and duration of signalling failures the network can tolerate without impacting on service levels.
Parliamentary reference - Report number #170 - released 15 August 2007
Dealing with household burglaries
The NSW Police Force has done well to maintain a downward trend in household burglaries in NSW since 2001. Police recognise the impact this crime has on the community’s perception of security and safety. Success has come from a continued focus on prevention and enforcement. However, there are other factors which have influenced this result. These include a rise in the rate and length of imprisonment, the heroin drought in 2001, increases in the number of heroin users entering treatment, low levels of unemployment, and increases in the average weekly earnings of young males.
Parliamentary reference - Report number #169 - released 27 June 2007
Connecting with public transport
We see considerable potential for the Ministry of Transport to plan and manage interchanges more effectively, so as to make better use of our public transport network. We believe that the Ministry now needs to focus more on multi-modal transport planning and interchange performance. It needs to assign responsibility for the coordination and oversight of inter-modal operations to an entity resourced for the purpose. Without this it will continue to be very difficult to identify and address unmet needs, seek and secure stakeholder funding, and monitor and evaluate system performance.
Parliamentary reference - Report number #168 - released 6 June 2007
Readiness to respond: Follow-up audit
The Ambulance Service of New South Wales has substantially implemented the 28 recommendations of the 2001 audit report that it accepted. It has also introduced significant new initiatives to improve performance that were not part of the 2001 recommendations. It has made substantial changes to its organisation and operations to implement these changes. Many of the changes are still proceeding. The Service has addressed a key finding of the 2001 audit report - that it did not have adequate, relevant or credible management data for decision making. The Service now has five years of operational data from the Computer Aided Dispatch (CAD) system.
Parliamentary reference - Report number #167 - released 6 June 2007
Using computers in schools for teaching and learning: Follow-up audit
We consider that students and teachers have significantly better access to computers than was the case when we did our 2000 audit. Teachers and students are also increasingly using computers for teaching and learning. The Department of Education and Training has been proactive in advancing Information Communication Technology (ICT) into schools. However we found that teachers and students are less likely to use computers if there are problems with infrastructure or technical support.
Parliamentary reference - Report number #166 - released 9 May 2007
