Reports
Progress and measurement of the Premier's Priorities
The Premier’s Implementation Unit uses a systematic approach to measuring and reporting progress towards the Premier’s Priorities performance targets, but public reporting needed to improve, according to a report released today by the Auditor-General of NSW, Margaret Crawford.
The Premier of New South Wales has established 12 Premier’s Priorities. These are key performance targets for government.
| The 12 Premier's Priorities | |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Source: Department of Premier and Cabinet, Premier’s Priorities website.
Each Premier’s Priority has a lead agency and minister responsible for achieving the performance target.
The Premier’s Implementation Unit (PIU) was established within the Department of Premier and Cabinet (DPC) in 2015. The PIU is a delivery unit that supports agencies to measure and monitor performance, make progress toward the Premier’s Priorities targets, and report progress to the Premier, key ministers and the public.
This audit assessed how effectively the NSW Government is progressing and reporting on the Premier's Priorities.
The Premier’s Implementation Unit (PIU) is effective in assisting agencies to make progress against the Premier’s Priorities targets. Progress reporting is regular but transparency to the public is weakened by the lack of information about specific measurement limitations and lack of clarity about the relationship of the targets to broader government objectives.The PIU promotes a systematic approach to measuring performance and reporting progress towards the Premier’s Priorities’ performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives.
The PIU provides a systematic approach to measuring performance and reporting progress towards the Premier's Priorities performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives. The data used to measure the Premier’s Priorities comes from a variety of government and external datasets, some of which have known limitations. These limitations are not revealed in public reporting, and only some are revealed in progress reported to the Premier and ministers. This limits the transparency of reporting.
The PIU assists agencies to avoid unintended outcomes that can arise from prioritising particular performance measures over other areas of activity. The PIU has adopted a collaborative approach to assisting agencies to analyse performance using data, and helping them work across organisational silos to achieve the Premier’s Priorities targets.
Data used to measure progress for some of the Premier’s Priorities has limitations which are not made clear when progress is reported. This reduces transparency about the reported progress. Public reporting would also be improved with additional information about the relationship between specific performance measures and broader government objectives.
The PIU is responsible for reporting progress to the Premier, key ministers and the public. Agencies provide performance data and some play a role in preparing progress reports for the Premier and ministers. For 11 of the Premier's Priorities, progress is reported against measurable and time-related performance targets. For the infrastructure priority, progress is reported against project milestones.
Progress of some Priorities is measured using data that has known limitations, which should be noted wherever progress is reported. For example, the data used to report on housing completions does not take housing demolitions into account, and is therefore overstating the contribution of this performance measure to housing supply. This known limitation is not explained in progress reports or on the public website.
Data used to measure progress is sourced from a mix of government and external datasets. Updated progress data for most Premier’s Priorities is published on the Premier’s Priorities website annually, although reported to the Premier and key ministers more frequently. The PIU reviews the data and validates it through fieldwork with front line agencies. The PIU also assists agencies to avoid unintended outcomes that can arise from prioritising single performance measures. Most, but not all, agencies use additional indicators to check for misuse of data or perverse outcomes.
We examined the reporting processes and controls for five of the Premier’s Priorities. We found that there is insufficient assurance over the accuracy of the data on housing approvals.
The relationships between performance measures and broader government objectives is not always clearly explained on the Premier’s Priority website, which is the key source of public information about the Premier’s Priorities. For example, the Premier’s Priority to reduce litter volumes is communicated as “Keeping our Environment Clean.” While the website explains why reducing litter is important, it does not clearly explain why that particular target has been chosen to measure progress in keeping the environment clean.
By December 2018, the Department of Premier and Cabinet should:
- improve transparency of public reporting by:
- providing information about limitations of reported data and associated performance
- clarifying the relationship between the Premier’s Priorities performance targets and broader government objectives.
- ensure that processes to check and verify data are in place for all agency data sources
- encourage agencies to develop and implement additional supporting indicators for all Premier’s Priority performance measures to prevent and detect unintended consequences or misuse of data.
The Premier's Implementation Unit is effective in supporting agencies to deliver progress towards the Premier’s Priority targets.
The PIU promotes a systematic approach to monitoring and reporting progress against a target, based on a methodology used in delivery units elsewhere in the world. The PIU undertakes internal self-evaluation, and commissions regular reviews of methodology implementation from the consultancy that owns the methodology and helped to establish the PIU. However, the unit lacks periodic independent reviews of their overall effectiveness. The PIU has adopted a collaborative approach and assists agencies to analyse performance using data, and work across organisational silos to achieve the Premier’s Priorities targets.
Agency representatives recognise the benefits of being responsible for a Premier's Priority and speak of the value of being held to account and having the attention of the Premier and senior ministers.
By June 2019, the Department of Premier and Cabinet should:
- establish routine collection of feedback about PIU performance including:
- independent assurance of PIU performance
- opportunity for agencies to provide confidential feedback.
Appendix one: Response from agency
Appendix three: About the audit
Appendix four: Performance auditing
Parliamentary reference - Report number #307 - released 13 September 2018
Matching skills training with market needs
In 2012, governments across Australia entered into the National Partnership Agreement on Skills Reform. Under the National Partnership Agreement, the Australian Government provided incentive payments to States and Territories to move towards a more contestable Vocational Education and Training (VET) market. The aim of the National Partnership Agreement was to foster a more accessible, transparent, efficient and high quality training sector that is responsive to the needs of students and industry.
The New South Wales Government introduced the Smart and Skilled program in response to the National Partnership Agreement. Through Smart and Skilled, students can choose a vocational course from a list of approved qualifications and training providers. Students pay the same fee for their chosen qualification regardless of the selected training provider and the government covers the gap between the student fee and the fixed price of the qualification through a subsidy paid to their training provider.
Smart and Skilled commenced in January 2015, with the then Department of Education and Communities having primary responsibility for its implementation. Since July 2015, the NSW Department of Industry (the Department) has been responsible for VET in New South Wales and the implementation of Smart and Skilled.
The NSW Skills Board, comprising nine part-time members appointed by the Minister for Skills, provides independent strategic advice on VET reform and funding. In line with most other States and Territories, the Department maintains a 'Skills List' which contains government subsidised qualifications to address identified priority skill needs in New South Wales.
This audit assessed the effectiveness of the Department in identifying, prioritising, and aligning course subsidies to the skill needs of NSW. To do this we examined whether:
- the Department effectively identifies and prioritises present and future skill needs
- Smart and Skilled funding is aligned with the priority skill areas
- skill needs and available VET courses are effectively communicated to potential participants and training providers.
Smart and Skilled is a relatively new and complex program, and is being delivered in the context of significant reform to VET nationally and in New South Wales. A large scale government funded contestable market was not present in the VET sector in New South Wales before the introduction of Smart and Skilled. This audit's findings should be considered in that context.
The Department needs to better use the data it has, and collect additional data, to support its analysis of priority skill needs in New South Wales, and direct funding accordingly.
- funding scholarships and support for disadvantaged students
- funding training in regional and remote areas
- providing additional support to deliver some qualifications that the market is not providing.
The Department needs to evaluate these funding strategies to ensure they are achieving their goals. It should also explore why training providers are not delivering some priority qualifications through Smart and Skilled.
Training providers compete for funding allocations based on their capacity to deliver. The Department successfully manages the budget by capping funding allocated to each Smart and Skilled training provider. However, training providers have only one year of funding certainty at present. Training providers that are performing well are not rewarded with greater certainty.
The Department needs to improve its communication with prospective students to ensure they can make informed decisions in the VET market.
The Department also needs to communicate more transparently to training providers about its funding allocations and decisions about changes to the NSW Skills List.
The Department relies on stakeholder proposals to update the NSW Skills List. Stakeholders include industry, training providers and government departments. These stakeholders, particularly industry, are likely to be aware of skill needs, and have a strong incentive to propose qualifications that address these needs. The Department’s process of collecting stakeholder proposals helps to ensure that it can identify qualifications needed to address material skill needs.
It is also important that the Department ensures the NSW Skills List only includes priority qualifications that need to be subsidised by government. The Department does not have robust processes in place to remove qualifications from the NSW Skills List. As a result, there is a risk that the list may include lower priority skill areas. Since the NSW Skills List was first created, new additions to the list have outnumbered those removed by five to one.
The Department does not always validate information gathered from stakeholder proposals, even when it has data to do so. Further, its decision making about what to include on, or delete from, the NSW Skills List is not transparent because the rationale for decisions is not adequately documented.
The Department is undertaking projects to better use data to support its decisions about what should be on the NSW Skills List. Some of these projects should deliver useful data soon, but some can only provide useful information when sufficient trend data is available.
Recommendation
The Department should:
- by June 2019, increase transparency of decisions about proposed changes to the NSW Skills List and improve record-keeping of deliberations regarding these changes
- by December 2019, use data more effectively and consistently to ensure that the NSW Skills List only includes high priority qualifications
Only qualifications on the NSW Skills List are eligible for subsidies under Smart and Skilled. As the Department does not have a robust process for removing low priority qualifications from the NSW Skills list, some low priority qualifications may be subsidised.
The Department allocates the Smart and Skilled budget through contracts with Smart and Skilled training providers. Training providers that meet contractual obligations and perform well in terms of enrolments and completion rates are rewarded with renewed contracts and more funding for increased enrolments, but these decisions are not based on student outcomes. The Department reduces or removes funding from training providers that do not meet quality standards, breach contract conditions or that are unable to spend their allocated funding effectively. Contracts are for only one year, offering training providers little funding certainty.
Smart and Skilled provides additional funding for scholarships and for training providers in locations where the cost of delivery is high or to those that cater to students with disabilities. The Department has not yet evaluated whether this additional funding is achieving its intended outcomes.
Eight per cent of the qualifications that have been on the NSW Skills List since 2015 are not delivered under Smart and Skilled anywhere in New South Wales. A further 14 per cent of the qualifications that are offered by training providers have had no student commencements. The Department is yet to identify the reasons that these high priority qualifications are either not offered or not taken up by students.
Recommendation
The Department should:
- by June 2019, investigate why training providers do not offer, and prospective students do not enrol in, some Smart and Skilled subsidised qualifications
- by December 2019, evaluate the effectiveness of Smart and Skilled funding which supplements standard subsidies for qualifications on the NSW Skills List, to determine whether it is achieving its objectives
- by December 2019, provide longer term funding certainty to high performing training providers, while retaining incentives for them to continue to perform well.
In a contestable market, it is important for consumers to have sufficient information to make informed decisions. The Department does not provide some key information to prospective VET students to support their decisions, such as measures of provider quality and examples of employment and further education outcomes of students completing particular courses. Existing information is spread across numerous channels and is not presented in a user friendly manner. This is a potential barrier to participation in VET for those less engaged with the system or less ICT literate.
The Department conveys relevant information about the program to training providers through its websites and its regional offices. However, it could better communicate some specific information directly to individual Smart and Skilled training providers, such as reasons their proposals to include new qualifications on the NSW Skills List are accepted or rejected.
While the Department is implementing a communication strategy for VET in New South Wales, it does not have a specific communications strategy for Smart and Skilled which comprehensively identifies the needs of different stakeholders and how these can be addressed.
Recommendation
By December 2019, the Department should develop and implement a specific communications strategy for Smart and Skilled to:
- support prospective student engagement and informed decision making
- meet the information needs of training providers
Appendix one - Response from agency
Appendix two - About the audit
Appendix three - Performance auditing
Parliamentary reference - Report number #305 - released 26 July 2018
Regulation of water pollution in drinking water catchments and illegal disposal of solid waste
There are important gaps in how the Environmental Protection Authority (EPA) implements its regulatory framework for water pollution in drinking water catchments and illegal solid waste disposal. This limits the effectiveness of its regulatory responses, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford.
| By 31 December 2018, to improve governance and oversight, the EPA should: | |
| 1. | implement a more effective performance framework with regular reports to the Chief Executive Officer and to the EPA Board on outcomes-based key result areas that assess its environmental and regulatory performance and trends over time |
| By 30 June 2019, to improve consistency in its practices, the EPA should: | |
| 2. | progressively update and make accessible its policies and procedures for regulatory operations, and mandate procedures where necessary to ensure consistent application |
| 3. | implement internal controls to monitor the consistency and quality of its regulatory operations. |
| By 30 June 2019, to address worsening water quality in Lake Burragorang, the EPA should: | |
| 4. | (a) review the impact of its licensed activities on water quality in Lake Burragorang, and |
| (b) develop strategies relating to its licensed activities (in consultation with other relevant NSW Government agencies) to improve and maintain the lake's water quality. | |
| To improve compliance monitoring, the EPA should implement procedures to: | |
| 5. | by 30 June 2019, validate self-reported information, eliminate hardcopy submissions and require licensees to report on their breaches of the Act and associated regulations in their annual returns |
| 6. | by 31 December 2018, conduct mandatory site inspections under the risk-based licensing scheme to assess compliance with all regulatory requirements and licence conditions. |
| By 31 December 2018 to improve enforcement, the EPA should: | |
| 7. | Implement procedures to systematically assess non-compliances with licence conditions and breaches of the Act and to implement appropriate and consistent regulatory actions. |
Appendix one – Response from agency
Appendix two – List of enforcement tools
Appendix three – The EPA's organisational structure
Appendix four – The EPA's regions and branches
Appendix five – About the audit
Appendix six – Performance auditing
Parliamentary reference - Report number #304 - released 28 June 2018
Fraud controls in local councils
Many local councils need to improve their fraud control systems, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford. The report highlights that councils often have fraud control procedures and systems in place, but are not ensuring people understand them and how they work. There is also significant variation between councils in the quality of their fraud controls.
Fraud can directly influence councils’ ability to deliver services, and undermine community confidence and trust. ICAC investigations, such as the recent Operation Ricco into the former City of Botany Bay Council, show the financial and reputational damage that major fraud can cause. Good fraud control practices are critical for councils and the community.
The Audit Office of New South Wales 2015 Fraud Control Improvement Kit (the Kit) aligns with the Fraud and Corruption Control Standard AS8001-2008 and identifies ten attributes of an effective fraud control system. This audit used the Kit to assess how councils manage the risk of fraud. It identifies areas where fraud control can improve.
Fraud can disrupt the delivery and quality of services and threaten the financial stability of councils.
Recent reviews of local government in Queensland and Victoria identify that councils are at risk of fraud because they purchase large quantities of goods and services using devolved decision making arrangements. The Queensland Audit Office in its 2014–15 report 'Fraud Management in Local Government' found that ‘Councils are exposed to high-risks of fraud and corruption because of the high volume of goods and services they procure, often from local suppliers; and because of the high degree of decision making vested in councils'. They also highlight some common problems faced by councils including the absence of fraud control plans and failure to conduct regular reviews of their internal controls. Also, in 2008 and 2012 the Victorian Auditor-General identified the importance of up-to-date fraud control planning, clearly documented related policies, training staff to identify fraud risks and the importance of controls such as third party management.
Investigations into councils by the NSW Independent Commission Against Corruption (ICAC), such as the recent Operation Ricco, show the impact that fraud can have on councils. These impacts include significant financial loss, and negative public perceptions about how well councils manage fraud. The findings of these investigations also show the importance of good fraud controls for councils.
|
Operation Ricco In its report on Operation Ricco, the ICAC found that the Chief Financial Officer (CFO) of the City of Botany Bay Council and others dishonestly exercised official functions to obtain financial benefits for themselves and others by causing fraudulent payments from the Council for their benefit. It also identified the CFO received inducements for favourable treatment of contractors. The report noted that there were overwhelming failures in the council’s procedures and governance framework that created significant opportunities for corruption, of which the CFO and others took advantage. It found weaknesses across a wide variety of governance processes and functions, including those involving the general manager, the internal audit function, external audit, and the operation of the audit committee. |
The strength of fraud control systems varies significantly across New South Wales local councils, and many councils we surveyed need to improve significantly.
Most surveyed councils do not have fraud control plans that direct resources to mitigating the specific fraud risks they face. Few councils reported that they conduct regular risk assessments or health checks to ensure they respond effectively to the risks they identify.
There are sector wide weaknesses that impact on the strength of councils' fraud control practice. Less than one-third of councils that responded to the survey:
- communicate their expectations about ethical conduct and responsibility for fraud control to staff
- regularly train staff to identify and respond to suspected fraud
- inform staff or the wider community how to report suspected fraud and how reports made will be investigated.
The audit also identified a pattern of councils developing policies, procedures or systems without ensuring people understand them, or assessing that they work. This reduces the likelihood that staff will actually use them.
In general, metropolitan and regional councils surveyed have stronger fraud control systems than rural councils.
Newly amalgamated councils are operating with systems inherited from two or more pre-amalgamated councils. These councils are developing new systems for their changed circumstances.
Five councils surveyed reported that they did not comply with the Public Interest Disclosure Act 1994.
Observations for the sector:
Councils should improve their fraud controls by:
- tailoring fraud control plans to their circumstances and specific risks
- systematically and regularly reviewing their fraud risks and fraud control systems to keep their plans up to-date
- effectively communicating fraud risks, and how staff and the community can report suspected fraud
- ensuring that they comply with the Public Interest Disclosure Act 1994.
Recommendation:
That the Office of Local Government:
- work with councils to ensure they comply with the Public Interest Disclosure Act 1994.
That the Office of Local Government:
- work with state entities and councils to develop a common approach to how fraud complaints and incidences are defined and categorised so that they can:
- better use data to provide a clearer picture of the level of fraud within councils
- measure the effectiveness of, and drive improvement in councils' fraud controls systems
Appendix one – Response from agency
Appendix three – About the audit
Appendix four – Performance auditing
Parliamentary reference - Report number #303 - released 22 June 2018
Managing risks in the NSW public sector: risk culture and capability
The Ministry of Health, NSW Fair Trading, NSW Police Force, and NSW Treasury Corporation are taking steps to strengthen their risk culture, according to a report released today by the Auditor-General, Margaret Crawford. 'Senior management communicates the importance of managing risk to their staff, and there are many examples of risk management being integrated into daily activities', the Auditor-General said.
We did find that three of the agencies we examined could strengthen their culture so that all employees feel comfortable speaking openly about risks. To support innovation, senior management could also do better at communicating to their staff the levels of risk they are willing to accept.
Effective risk management is essential to good governance, and supports staff at all levels to make informed judgements and decisions. At a time when government is encouraging innovation and exploring new service delivery models, effective risk management is about seizing opportunities as well as managing threats.
Over the past decade, governments and regulators around the world have increasingly turned their attention to risk culture. It is now widely accepted that organisational culture is a key element of risk management because it influences how people recognise and engage with risk. Neglecting this ‘soft’ side of risk management can prevent institutions from managing risks that threaten their success and lead to missed opportunities for change, improvement or innovation.
This audit assessed how effectively NSW Government agencies are building risk management capabilities and embedding a sound risk culture throughout their organisations. To do this we examined whether:
- agencies can demonstrate that senior management is committed to risk management
- information about risk is communicated effectively throughout agencies
- agencies are building risk management capabilities.
The audit examined four agencies: the Ministry of Health, the NSW Fair Trading function within the Department of Finance, Services and Innovation, NSW Police Force and NSW Treasury Corporation (TCorp). NSW Treasury was also included as the agency responsible for the NSW Government's risk management framework.
In assessing an agency’s risk culture, we focused on four key areas:
Executive sponsorship (tone at the top)
In the four agencies we reviewed, senior management is communicating the importance of managing risk. They have endorsed risk management frameworks and funded central functions tasked with overseeing risk management within their agencies.
That said, we found that three case study agencies do not measure their existing risk culture. Without clear measures of how employees identify and engage with risk, it is difficult for agencies to tell whether employee's behaviours are aligned with the 'tone' set by the executive and management.
For example, in some agencies we examined we found a disconnect between risk tolerances espoused by senior management and how these concepts were understood by staff.
Employee perceptions of risk management
Our survey of staff indicated that while senior leaders have communicated the importance of managing risk, more could be done to strengthen a culture of open communication so that all employees feel comfortable speaking openly about risks. We found that senior management could better communicate to their staff the levels of risk they should be willing to accept.
Integration of risk management into daily activities and links to decision-making
We found examples of risk management being integrated into daily activities. On the other hand, we also identified areas where risk management deviated from good practice. For example, we found that corporate risk registers are not consistently used as a tool to support decision-making.
Support and guidance to help staff manage risks
Most case study agencies are monitoring risk-related skills and knowledge of their workforce, but only one agency has addressed the gaps it identified. While agencies are providing risk management training, surveyed staff in three case study agencies reported that risk management training is not adequate.
NSW Treasury provides agencies with direction and guidance on risk management through policy and guidelines. In line with better practice, NSW Treasury's principles-based policy acknowledges that individual agencies are in a better position to understand their own risks and design risk management frameworks that address those risks. Nevertheless, there is scope for NSW Treasury to refine its guidance material to support a better risk culture in the NSW public sector.
Recommendation
By May 2019, NSW Treasury should:
- Review the scope of its risk management guidance, and identify additional guidance, training or activities to improve risk culture across the NSW public sector. This should focus on encouraging agency heads to form a view on the current risk culture in their agencies, identify desirable changes to that risk culture, and take steps to address those changes.
Appendix one - Response from agencies
Appendix three - About the audit
Appendix four - Performance auditing
Parliamentary reference - Report number #298 - released 23 April 2018
Managing IT Services Contracts
Neither agency (NSW Ministry of Health and NSW Police Force) demonstrated that they continued to get value for money over the life of these long term contracts or that they had effectively managed all critical elements of the three contracts we reviewed post award. This is because both agencies treated contract extensions or renewals as simply continuing previous contractual arrangements, rather than as establishing a new contract and financial commitment. Consequently, there was not a robust analysis of the continuing need for the mix and quantity of services being provided or an assessment of value for money in terms of the prices being paid.
Parliamentary reference - Report number #220 - released 1 February 2012
Police Rostering
Overall, the NSW Police Force (the Force) is not meeting its target for responding to urgent calls for assistance. Today, one in four complaints is about customer service. Victims of crime report issues including difficulties in contacting an officer and frustration at having to wait for an officer to return to duty to find out about their case. The timeliness of investigations is also at risk when follow-up is delayed. Seventy per cent of commanders surveyed indicated that rosters did not support continuity of investigations or victims’ needs.
We are also concerned that the current roster pattern may not be in the best interest of the health and safety of officers. Sick and injury leave is higher in NSW than other states and the Force is yet to assess the extent to which rosters may contribute to this. The Force also needs to do more to protect its general duties officers from fatigue in line with contemporary practice for managing shiftwork.
Parliamentary reference - Report number #173 - released 5 December 2007
Signal failures on the metropolitan rail network
Between 2004 and 2006, the number of signalling failures, signalling downtime and the number of trains delayed as a result of signal failures all fell. RailCorp’s on-time running performance improved over the same period. The fall in failures is a clear indication of improved performance. Changes in the definition of on-time and to the timetable during 2005 and 2006 however make it difficult to determine whether improvements in response downtime and signalling delays are due to a true performance improvement. To build upon this strong base, RailCorp needs to determine with more confidence the number and duration of signalling failures the network can tolerate without impacting on service levels.
Parliamentary reference - Report number #170 - released 15 August 2007
Dealing with household burglaries
The NSW Police Force has done well to maintain a downward trend in household burglaries in NSW since 2001. Police recognise the impact this crime has on the community’s perception of security and safety. Success has come from a continued focus on prevention and enforcement. However, there are other factors which have influenced this result. These include a rise in the rate and length of imprisonment, the heroin drought in 2001, increases in the number of heroin users entering treatment, low levels of unemployment, and increases in the average weekly earnings of young males.
Parliamentary reference - Report number #169 - released 27 June 2007
Addressing the needs of young offenders
The Department of Juvenile Justice and the NSW Police Force generally work together and with other agencies to identify and address the needs of young offenders. We found that this is done particularly well for young offenders who end up in detention and under community supervision. Both the Department of Juvenile Justice and the NSW Police Force have put some practices in place to identify and address the needs of young offenders diverted from the Children’s Court. However, we found that they are not doing this systematically.
Parliamentary reference - Report number #164 - released 28 March 2007
