The following is the website banner for the Auditor-General for New South Wales report on Progress and measurement of the Premier


  

Progress and measurement of the Premier's Priorities
 Download full report (PDF) - Download media release (PDF)

 

Contents

Executive Summary -  1. Introduction -  2. Progress Reporting 

3. The effectiveness of the delivery unit -  Appendices

 

  

Executive Summary

The Premier of New South Wales has established 12 Premier’s Priorities. These are key performance targets for government.

Exhibit 1: The 12 Premier’s Priorities

The 12 Premier's Priorities
  • 150,000 new jobs by 2019
  • Reduce the volume of litter by 40 per cent by 2020
  • 10 key projects in metro and regional areas to be delivered on time and on budget, and nearly 90 local infrastructure projects to be delivered on time
  • Increase the proportion of NSW students in the top two NAPLAN bands by eight per cent by 2019
  • Increase the proportion of women in senior leadership roles in the NSW Government sector from 33 to 50 per cent by 2025 and double the number of Aboriginal and Torres Strait Islander people in senior leadership roles in the NSW Government sector, from 57 to 114
  • Increase the proportion of young people who successfully move from Specialist Homelessness Services to long-term accommodation to more than 34 per cent by 2019
  • 61,000 housing completions on average per year to 2021
  • Reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021
  • Improve customer satisfaction with key government services every year, this term of government to 2019
  • Decrease the percentage of children and young people re-reported at risk of significant harm by 15 per cent by 2020
  • 81 per cent of patients through emergency departments within four hours by 2019
  • Reduce overweight and obesity rates of children by five percentage points by 2025

Source: Department of Premier and Cabinet, Premier’s Priorities website.

Each Premier’s Priority has a lead agency and minister responsible for achieving the performance target.

The Premier’s Implementation Unit (PIU) was established within the Department of Premier and Cabinet (DPC) in 2015. The PIU is a delivery unit that supports agencies to measure and monitor performance, make progress toward the Premier’s Priorities targets, and report progress to the Premier, key ministers and the public.

This audit assessed how effectively the NSW Government is progressing and reporting on the Premier's Priorities.

The Premier’s Implementation Unit (PIU) is effective in assisting agencies to make progress against the Premier’s Priorities targets. Progress reporting is regular but transparency to the public is weakened by the lack of information about specific measurement limitations and lack of clarity about the relationship of the targets to broader government objectives.

The PIU promotes a systematic approach to measuring performance and reporting progress towards the Premier’s Priorities’ performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives. The data used to measure the Premier’s Priorities comes from a variety of government and external datasets, some of which have known limitations. These limitations are not revealed in public reporting, and only some are revealed in progress reported to the Premier and ministers. This limits the transparency of reporting.

The PIU assists agencies to avoid unintended outcomes that can arise from prioritising particular performance measures over other areas of activity. The PIU has adopted a collaborative approach to assisting agencies to analyse performance using data, and helping them work across organisational silos to achieve the Premier’s Priorities targets

1. Key findings

The Premier’s Priorities website should be more transparent about limitations in performance data

The data used to measure the Premier’s Priorities comes from a variety of existing government and external datasets, some of which have known limitations. For example, data used to measure “61,000 housing completions on average per year” uses data from the Australian Bureau of Statistics (ABS) that does not include housing demolitions, meaning it does not give an overall picture of additional available housing.

Some Premier’s Priorities performance measures are difficult to interpret, particularly those which are measured as proportions. For example, the domestic violence reoffending target is to “reduce the proportion of domestic violence reoffending by 25 per cent by 2021.” Progress towards this target could occur either through reduced reoffending relative to offending, or by increased offending relative to reoffending. In other words, the number of reoffences could increase or decrease, but it is not clear from the reported progress which is the case.

The approach of using existing data collection systems enabled agencies and the PIU to quickly begin measuring and reporting on performance rather than waiting on new data to be collected. However, the limitations in the data used should be disclosed wherever progress is reported, as should any additional information a reader needs to correctly interpret performance. This is not currently occurring in public reporting, and in some cases the data limitations are not included in progress reports provided to the Premier and ministers.

The housing approvals data has substantial limitations

The Premier’s Priorities target for “61,000 housing completions on average each year to 2021” has two supporting targets. The supporting target of “90 per cent of housing approvals determined within 40 days by 2019” has several limitations that should be addressed where feasible, or at least clearly communicated whenever performance is reported:

  • None of the housing approvals data sent to the Department of Planning and Environment (DPE) by councils is audited. In addition, some councils do not provide complete information to DPE about the number of days that private certifiers take to approve dwellings, so DPE uses an estimate based on self‑reported survey data from private certifiers.
  • The data does not measure duration from application to determination time as experienced by citizens. The reported time taken only includes time spent by council working on the development application and does not include time spent by councils waiting for additional information needed to determine the outcome of the application. The measure therefore understates the amount of time an applicant will have to wait for a determination.
Four of the five Premier’s Priorities we examined use additional indicators to detect unintended consequences of measurement

Agencies and the PIU are aware of the potential for performance targets to deliver unintended consequences such as reduced performance in areas not being monitored, or misuse of data to make performance look better than it is. We examined five of the Premier’s Priorities and found that four use additional indicators to monitor unusual or inconsistent performance that might indicate misuse of data or reduced performance in other areas. The housing approvals data is not supported by any additional indicators.

The PIU maintains a Premier’s Priorities website, but more information is needed about the performance targets

The Premier’s Priorities website is the main way information is communicated to the public about progress on the Premier’s Priorities, with the exception of the ten key infrastructure projects – for which progress is not reported on the website.

Each Premier’s Priority is labelled on the website with a short heading that describes a broad policy objective, however it is not clear in every case how reported targets are helping achieve this policy objective.

For example, the Premier’s Priority to “reduce the volume of litter by 40 per cent” has the heading “Keeping our Environment Clean.” Keeping the environment clean is a broader objective than can only be partially delivered through the specific target of reducing litter volumes.

The Premier’s Priorities website does not explain why each performance measure and target is important in the broader context of government priorities, or why particular targets have been selected to demonstrate progress in these areas.

Agencies report that the PIU provides effective support in progressing targets

The PIU’s role is to support delivery of government priorities by working with agencies to drive performance towards the Premier’s Priorities performance targets. The PIU provides independent oversight of reported performance and works with agencies to report regularly to the Premier, key ministers and the public. The PIU uses a systematic approach to support agencies and report progress, based on a recognised methodology. While the unit undertakes internal self-evaluation, and commissions regular reviews from the consultancy that helped to establish the PIU, the unit lacks periodic independent scrutiny.

Agencies report that they value the PIU’s:

  • collaborative approach
  • use of data to analyse performance and identify improvement opportunities
  • ability to work across organisational silos
  • communication skills.

 

2. Recommendations

By December 2018, the Department of Premier and Cabinet should:

  1. improve transparency of public reporting by:
    • providing information about limitations of reported data and associated performance
    • clarifying the relationship between the Premier’s Priorities performance targets and broader government objectives.
  1. ensure that processes to check and verify data are in place for all agency data sources
  2. encourage agencies to develop and implement additional supporting indicators for all Premier’s Priority performance measures to prevent and detect unintended consequences or misuse of data.

By June 2019, the Department of Premier and Cabinet should:

  1. establish routine collection of feedback about PIU performance including:
    • independent assurance of PIU performance
    • opportunity for agencies to provide confidential feedback.

 

 
Return to top



1. Introduction

1.1 Background

Development of the Premier's Priorities

Following the 2015 New South Wales state election, the then Premier identified 12 priorities for the government, each represented by a performance target (Exhibit 2). To drive delivery of these priorities, the Premier's Implementation Unit (PIU) was established within the Department of Premier and Cabinet (DPC).

The former Premier nominated three of the Priorities for closer scrutiny and greater involvement of the PIU. These are known as the ‘deep dives’ and are highlighted in red in Exhibit 2.

The current Premier recommitted to the Premier’s Priorities on taking office in early 2017, and made small adjustments to some existing targets in July 2017. An additional target measure was introduced to measure delivery of local infrastructure projects. A new target for housing completions was introduced while the original housing approvals target, and an additional rezoning target, were added as supporting target measures.

Exhibit 2: The Premier's Priorities with ‘deep dives’ highlighted, and lead agencies

Premier's Priority performance target Lead agency
150,000 new jobs by 2019 Department of Industry
Deliver ten key infrastructure projects in metro and regional areas on time and on budget by the end of 2024 Infrastructure NSW
Manage more than 80 local infrastructure projects across NSW to ensure they are successfully delivered or on track to be delivered by 2019 Regional NSW (DPC)
Increase the proportion of women in senior leadership roles in the NSW Government sector from 33 to 50 per cent by 2025 and double the number of Aboriginal and Torres Strait Islander people in senior leadership roles in the NSW Government sector from 57 to 114 by 2025 Public Service Commission
Increase the proportion of NSW students in the top two NAPLAN bands by eight per cent by 2019 Department of Education
Improve customer satisfaction with key government services every year, this term of government to 2019 Department of Finance, Services and Innovation
81 per cent of patients through emergency departments within four hours by 2019 Ministry of Health
Reduce the volume of litter by 40 per cent, by 2020 Environmental Protection Authority

61,000 housing completions on average per year to 2021:

  • 90 per cent of housing approvals determined within 40 days by 2019
  • State‑led rezoning for 10,000 additional dwellings on average per year in appropriate areas to 2021.
Department of Planning
Decrease the percentage of children and young people re‑reported at risk of significant harm by 15 per cent by 2021 Department of Family and Community Services
Reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021 Department of Justice
Increase the proportion of young people who successfully move from Specialist Homelessness Services to long‑term accommodation to more than 34 per cent by 2019 Department of Family and Community Services
Reduce overweight and obesity rates of children by five percentage points by 2025 Ministry of Health

Source: Department of Premier and Cabinet.

The role of the Premier's Implementation Unit

Key functions of the PIU are to work with agencies to ensure that implementation plans are in place for the Premier’s Priorities, to monitor, assess and report on progress, and to support agencies to address problems affecting delivery of the Premier’s Priorities. The PIU has adopted a delivery methodology from the United Kingdom that has been used in other delivery units around the world.

Monitoring and reporting progress

While agencies are responsible for achieving targets, the PIU monitors and reports on progress with agency input:

  • The Premier’s Priorities website reports progress to the public
  • Monthly Notes are provided to the Premier. These give an overall summary of performance and progress towards the targets, with more detailed information about the performance of deep dive priorities and selected other priorities
  • Delivery Reports are provided to the Premier and Cabinet every six months and report on the performance of all 12 Premier’s Priorities, including progress towards the targets
  • Stocktake meetings are detailed status meetings held every three months for deep dive priorities and are attended by the Premier, relevant ministers and secretaries, the lead deputy secretary for the Priority and PIU representatives.

 

1.2 Audit approach

This audit assessed how effectively the NSW Government is progressing and reporting on the Premier's Priorities. To do so, we examined whether the PIU accurately reports relevant and timely information regarding progress on the Premier's Priorities and is effective at supporting agencies to advance the Premier's Priorities.

The audit includes assessment of all 12 of the Premier's Priorities and examined five of the Premier’s Priorities in greater detail. See Appendix 3 for more details about the audit scope, criteria and methodology.

 

Return to top

 

2. Progress reporting

Data used to measure progress for some of the Premier’s Priorities has limitations which are not made clear when progress is reported. This reduces transparency about the reported progress. Public reporting would also be improved with additional information about the relationship between specific performance measures and broader government objectives.

The PIU is responsible for reporting progress to the Premier, key ministers and the public. Agencies provide performance data and some play a role in preparing progress reports for the Premier and ministers. For 11 of the Premier's Priorities, progress is reported against measurable and time-related performance targets. For the infrastructure priority, progress is reported against project milestones.

Progress of some Priorities is measured using data that has known limitations, which should be noted wherever progress is reported. For example, the data used to report on housing completions does not take housing demolitions into account, and is therefore overstating the contribution of this performance measure to housing supply. This known limitation is not explained in progress reports or on the public website.

Data used to measure progress is sourced from a mix of government and external datasets. Updated progress data for most Premier’s Priorities is published on the Premier’s Priorities website annually, although reported to the Premier and key ministers more frequently. The PIU reviews the data and validates it through fieldwork with front‑line agencies. The PIU also assists agencies to avoid unintended outcomes that can arise from prioritising single performance measures. Most, but not all, agencies use additional indicators to check for misuse of data or perverse outcomes.

We examined the reporting processes and controls for five of the Premier’s Priorities. We found that there is insufficient assurance over the accuracy of the data on housing approvals.

The relationships between performance measures and broader government objectives is not always clearly explained on the Premier’s Priority website, which is the key source of public information about the Premier’s Priorities. For example, the Premier’s Priority to reduce litter volumes is communicated as “Keeping our Environment Clean.” While the website explains why reducing litter is important, it does not clearly explain why that particular target has been chosen to measure progress in keeping the environment clean.

By December 2018, the Department of Premier and Cabinet should:

  1. improve transparency of public reporting by:
    • providing information about limitations of reported data and associated performance
    • clarifying the relationship between the Premier’s Priorities performance targets and broader government objectives.
  1. ensure that processes to check and verify data are in place for all agency data sources
  2. encourage agencies to develop and implement additional supporting indicators for all Premier’s Priority performance measures to prevent and detect unintended consequences or misuse of data.

2.1 Performance measures and targets

Progress of most Premier's Priorities is reported against measurable targets

The 11 non‑infrastructure Premier's Priorities comprise 14 performance targets, each with a target delivery date. The infrastructure priority (key metropolitan, regional and local infrastructure projects to be delivered on time and on budget) monitors the progress of ten key infrastructure projects and more than 80 local infrastructure projects against project milestones. The Premier’s Priorities website includes expected completion dates for the ten key infrastructure projects, but does not report progress of the projects. Additional information about progress is available for some key projects on the individual project websites. At the time of the audit, the public reporting for the local infrastructure projects was undergoing development and was planned to identify each project and its completion status.

Some performance measures are complex and difficult to interpret

Some performance, particularly that which is reported as a percentage or proportion, can be ambiguous and difficult to interpret.

An example is the measure for domestic violence reoffending. The Premier’s Priority is a target to “reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021.” Progress towards this target could occur through a reduction in reoffending compared to offending, or by an increase in offending compared to reoffending. In other words, the number of reoffences could increase or decrease and it is not clear from the reported rate of reoffending which is the case.

The agency and PIU reported that the intent of measuring reoffending is to reduce reoffending through interventions targeted at offenders. However, according to data from the Bureau of Crime Statistics and Research, around 21 per cent of reoffending occurs before an initial offence is ever reported. This means that both the original offence and the reoffence are reported at the same time, with no opportunity for Police or Corrections to intervene in between.  The limitations of this performance measure as an indication of progress are recognised by the Department of Justice and the PIU. These limitations are not included in public reporting, but are reported to the Premier.

Other Premier’s Priorities have similar issues, particularly those which are measured as proportions. Wherever performance is reported, information should be provided to assist with interpreting the performance data. The Premier’s Priority website currently provides some interpretative information about reported performance, but this information does not help to explain the reported percentages, which could be made clearer.

Public communication does not always reflect what is being measured

Information about how the Premier’s Priorities targets contribute to the broader government objectives would improve clarity and help the community to understand the complexity of making progress in some of the more challenging areas.

Most of the Premier’s Priorities are communicated to the public using broad objective statements that do not clearly communicate what is, in most cases, a single performance measure. For example, the Premier’s Priority for increasing the number of people who are moved through emergency departments in less than four hours is communicated as “Improving Service Levels in Hospitals”. Limited information is provided to the public about the relationship between the broader objectives and the Premier’s Priorities targets.

That said, some of the Premier’s Priorities targets align well with the objectives. An example is the Premier’s Priority to reduce overweight and obesity rates of children by five percentage points by 2015. This is communicated as “Tackling Childhood Obesity” which provides a clear indication of what the Premier’s Priority is trying to achieve.

On the other hand, the overall objective “Making Housing More Affordable” is used to communicate the Premier’s Priority of achieving an average of 61,000 housing completions per year to 2021. The housing completions priority has two supporting targets, 90 per cent of housing approvals determined within 40 days by 2019 and state-led rezoning for 10,000 new dwellings on average per year to 2021. Neither the housing completions target, nor either of the two supporting targets, are direct measures of housing affordability. A 2016 briefing paper presented to the Australian Parliament defines housing affordability as the relationship between expenditure on housing and household incomes. While more housing completions, faster approvals and increased rezoning can influence housing supply, improved performance in any of these areas will not necessarily lead to improved housing affordability. DPE staff advised us that a focus on housing supply is important to housing affordability and also something that the NSW Government is in a position to influence. While this may be the case, there is no explanation about the link between these performance measures and housing affordability provided on the Premier’s Priorities website.

Most agencies reported that they are doing more work to advance the broader government objective than is implied by the Premier’s Priorities performance targets. For example, the Ministry of Health monitors more than 60 performance indicators related to service levels in hospitals. The Premier's Priority performance measure, time spent by patients in the Emergency Department, is only one of these.

2.2 Performance data

A review of data sources identified potential to improve data collection and reporting for some Premier’s Priorities

Progress of the Premier’s Priorities is measured using a range of existing data sources. Some data is generated within the NSW Government, while other data is sourced from external organisations such as the ABS. The data sources for each of the Premier’s Priorities is included in Appendix 2.

In the second half of 2017, approximately 18 months after it was established, the PIU commissioned a review of the data used to report on the Premier’s Priorities. This review identified the potential to improve data collection and progress reporting for a number of the Premier’s Priorities. Our audit confirmed that the PIU and agencies are taking on board most of the key recommendations of the review through improved data collection or triangulation of progress through alternative data sources. One recommendation, enhancement of the youth homelessness data, is considered by the PIU to be out of scope for the Premier’s Priorities.

In addition to the commissioned review, the PIU regularly tests the quality of data by triangulating performance data through fieldwork with front line service staff and interrogating data to confirm reported performance.

Data limitations should be disclosed whenever progress is reported

Some of the data used to report progress has inherent limitations that should be made clear in progress reports and public‑facing materials. For example, reported performance for Domestic Violence Reoffending effectively lags by 24 months. Performance data for 2018 is reporting domestic violence that occurred first in 2016 with a reoffence in 2017. This time lag is noted on materials provided to the Premier, but is not clear in the material reported to the public.

Exhibit 3: Limitations in data used to report progress on housing affordability

The Premier’s Priority of 61,000 housing completions on average per year to 2021 is monitored using ABS survey data that is based on gross completions and does not take housing demolitions into account. 

A supporting measure is the length of time for housing approvals to be determined. This is a process measure that includes days that council is working on the development application. Councils can ‘stop the clock’ if they need more information from an applicant or another government agency, meaning that the applicant's experience of housing approval duration can be longer than the time reported against this indicator. PIU staff told us that councils have varied views about whether the 40-day target should represent the citizen experience.

In addition, the number of days to determine a housing approval does not include multi‑unit dwellings. DPE reported that housing approvals for multi‑unit dwellings represents only three per cent of development applications. However, the intent of this Premier's Priority is to improve housing affordability in part through increased housing supply. Over the past four years, multi‑unit dwellings represented almost 70 per cent of approved dwellings in the greater Sydney area. Excluding this proportion of the dwellings approved in the greater Sydney region diminishes the relevance of the reported progress.

These limitations are not mentioned in materials provided to the Premier, ministers or the public.

A house with a wood frame - under construction

The PIU and agencies are using data that is readily available, thus avoiding the need to create and maintain new datasets. While this approach means that additional data does not have to be collected to measure and monitor the Premier’s Priorities, the limitations of the existing datasets should be made transparent whenever progress is reported.

Using data from existing data sources also means that the PIU can only report progress when updated data is available. At the time of the audit, progress of around half of the Premier’s Priorities has not been updated on the website for more than six months, and most data is only updated annually. At the very least, the Premier’s Priorities website should include information about when the next progress update will be available.

Exhibit 4: Limitations of performance measures and performance data

Premier’s Priority performance target Issues identified by the Audit Office*
150,000 new jobs by 2019. Reported performance includes part-time jobs and unpaid jobs in family companies as per ABS definition, which is not made clear on the Premier’s Priority website.
Deliver ten key infrastructure projects in metro and regional areas on time and on budget by the end of 2024. Progress is not reported on the Premier’s Priorities website.
Manage more than 80 local infrastructure projects across NSW to ensure they are successfully delivered or on track to be delivered by 2019. Nil.

Increase the proportion of women in senior leadership roles in the NSW Government sector from 33 to 50 per cent by 2025.

Double the number of Aboriginal and Torres Strait Islander people in senior leadership roles in the NSW Government sector from 57 to 114 by 2025.

Communicated as 'Driving public sector diversity' with no explanation why only these two types of diversity are an appropriate focus.
Increase the proportion of NSW students in the top two NAPLAN bands by eight per cent by 2019. Communicated as 'Improving education results', with limited explanation about why only these two bands of two tests are an appropriate focus.
Improve customer satisfaction with key government services every year, this term of government to 2019. Monitors a sample of 23 government services without communicating which. No definition of what is a 'key' service.
Eighty-one per cent of patients through emergency departments within four hours by 2019. Communicated as 'Improving service levels in hospitals', with no explanation why only emergency departments are an appropriate focus.
Reduce the volume of litter by 40 per cent, by 2020. Communicated as 'Keeping our environment clean', with no explanation why only litter is an appropriate focus.
61,000 housing completions on average per year to 2021: Communicated as 'Improving housing affordability', with no explanation how housing supply contributes to affordability. Also, data does not take housing demolitions into account and this is not explained on the website.
  • Ninety per cent of housing approvals determined within 40 days by 2019.
No explanation about how housing supply contributes to affordability. Does not measure time as experienced by the citizen and does not include multi‑unit dwellings. Relies on data from councils without sufficient quality controls to ensure accuracy.
  • State‑led rezoning for 10,000 additional dwellings on average per year in appropriate areas to 2021.
No explanation why housing supply contributes to affordability.
Decrease the percentage of children and young people re‑reported at risk of significant harm by 15 per cent by 2021. Communicated as 'Protecting our kids', but no explanation why only the chosen risk category is an appropriate focus.
Reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021. Appropriately communicated as 'Reducing domestic violence reoffending', but insufficient explanation of indicator limitations.
Increase the proportion of young people who successfully move from Specialist Homelessness Services to long‑term accommodation to more than 34 per cent by 2019. No explanation why only Specialist Homelessness Services is an appropriate focus. No indication of how long young people stay in long-term accommodation.
Reduce overweight and obesity rates of children by five percentage points by 2025. Appropriately communicated as 'Tackling childhood obesity'.

* These issues were identified at the time of the audit.

Performance data is monitored for unintended consequences

The audit team investigated the potential for unintended outcomes amongst five Premier's Priorities selected for closer examination, such as reduced performance in areas that are not measured, or misuse of data. All agencies, and the PIU, understand the risk of unintended outcomes, especially as most Premier’s Priorities are only single performance measures, which can be problematic. Four of the five responsible agencies demonstrated that they actively monitor performance data for unintended outcomes. Additional measures are monitored alongside the published Premier's Priority measure to look for distortions in data that might indicate misuse of data or reduced performance in other areas.

Exhibit 5: Example of an additional indicator

Family and Community Services (FACS) monitors cohort size to ensure data is not being manipulated to meet performance targets

The Premier’s Priority to decrease the percentage of children and young people at risk of significant harm from being re reported within 12 months of their case being closed is monitored using two numbers:

  • denominator ‑ number of children reported at risk of significant harm whose cases are closed as 'plan goal achieved' in a 12-month period
  • numerator ‑ number of children from the group above who are re reported at risk of significant harm within the following 12 months.

In addition to controls over data entry at the front line, and other checks, FACS monitors the size of the cohort of children at risk of significant harm whose cases it closed because 'plan goal achieved', the denominator. A relatively consistent cohort size provides a level of comfort that data entry is not manipulated to meet performance targets. FACS also monitors the mix of case closure reasons to determine whether anything unexpected has occurred.

NSW Family and Community Services logo

One of the Premier's Priorities we reviewed does not use additional indicators to look for unintended outcomes. Time taken to determine housing approvals, one of the supporting performance measures for the housing completions target, has potential for unintended outcomes as councils can ‘stop the clock’ to wait for additional information from the applicant. Additional indicators should be monitored alongside the housing approvals determination time to ensure that ‘stop the clock’ time is not used unnecessarily.

Adequate data controls are not in place for one of the Premier’s Priorities that we examined

The audit team closely examined the reported performance for five of the Premier's Priorities:

  • increase the proportion of NSW students in the top two NAPLAN bands by eight per cent by 2019
  • 81 per cent of patients through emergency departments within four hours by 2019
  • 61,000 housing completions on average per year to 2021, and the two supporting measures for housing approvals and state-led rezoning
  • decrease the percentage of children and young people re‑reported at risk of significant harm by 15 per cent by 2021
  • reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021.

Our examination included checking how the data is verified and managed, checking procedures for reporting progress to the Premier and the public and, where we had access to data, reproducing the reported performance graphs.

Wherever possible, data used to measure performance should be collected and managed by a different organisation to that being measured. Where this is not practical, other internal assurance mechanisms should be in place to maintain the integrity of the performance data, such as separating the data collection function from the area being measured. This helps mitigate the risk that the data are modified to attempt to produce a more favourable result.

Performance of five of the 12 Premier's Priorities is measured using data produced by the same agency that is responsible for performance:

  • public sector diversity
  • emergency department treatment times
  • housing approvals determination time and state-led rezoning (the supporting measures for housing completions)
  • child protection re-reporting
  • childhood obesity.

Of these, we examined data and reporting processes associated with three: emergency department treatment times, child protection re-reporting and the supporting measures for housing completions. We found no evidence that data was misreported in the emergency department data or the child protection re-reporting data. Also, within the responsible agencies, front‑line service is delivered by different parts of the organisation to those responsible for reporting performance, creating a separation between performance delivery and performance monitoring.

The data that reports time taken to determine housing approvals is initially generated by local councils, who send summary annual reports of their dwelling approval activity to DPE. These reports sometimes include inaccurate or incomplete data, and are subject to manual checking by DPE and requests for corrections or additional information. Summary data is then sent to the PIU for reporting to the Premier and the public.

There is no independent review or audit of the data sent to DPE by the councils, and no independent verification of the manual checking work done by DPE, so we are unable to confirm that the housing approvals data is subject to adequate controls.

In addition, some data sent to DPE from councils is missing the number of days taken to approve a dwelling, and so DPE use an estimate that is based on self‑reported survey data from private certifiers. We were unable to determine how often this occurs. DPE staff informed us that this data is only incomplete in some cases where private certifiers determine housing approvals for complying development. It would be more transparent to report this data as an estimate rather than actual numbers of approvals, with the data gaps clearly identified.

 

Return to top
 

 

3. The effectiveness of the delivery unit

The Premier's Implementation Unit is effective in supporting agencies to deliver progress towards the Premier’s Priority targets.

The PIU promotes a systematic approach to monitoring and reporting progress against a target, based on a methodology used in delivery units elsewhere in the world. The PIU undertakes internal self-evaluation, and commissions regular reviews of methodology implementation from the consultancy that owns the methodology and helped to establish the PIU. However, the unit lacks periodic independent reviews of their overall effectiveness. The PIU has adopted a collaborative approach and assists agencies to analyse performance using data, and work across organisational silos to achieve the Premier’s Priorities targets.

Agency representatives recognise the benefits of being responsible for a Premier's Priority and speak of the value of being held to account and having the attention of the Premier and senior ministers.

By June 2019, the Department of Premier and Cabinet should:

  1. establish routine collection of feedback about PIU performance including:
    • independent assurance of PIU performance
    • opportunity for agencies to provide confidential feedback.

3.1 Role and approach of the delivery unit

The role of the PIU differs for each agency, depending on need

The PIU supports delivery in three ways:

  • supporting the Premier to drive delivery of government priorities
  • providing independent oversight of progress
  • working closely with agency implementation teams to support delivery.

All agencies provide performance data to the PIU, which the PIU uses to communicate progress:

  • reports are delivered to the Premier for all agencies responsible for the Premier's Priorities
  • the Premier's Priorities website includes progress information for each Priority
  • a performance dashboard is available to key implementation staff and reports up to date progress for all Priorities.

Beyond performance reporting, the level of additional support provided by the PIU varies across agencies depending on their need. For some agencies, the PIU takes a relatively hands‑off role. For others, the PIU is working with the agency staff on a weekly basis. In one case, the PIU seconded staff into an agency to work on the rollout of a high-profile program. Most of the agencies we consulted said that the level of support they were receiving matched their needs and had been determined through a consultative process.

The PIU has adopted a methodology used by other delivery units

The approach adopted by the PIU is based on the Deliverology methodology, developed within the UK Government during the Blair administration, and adopted by delivery units around the world.

In summary, the methodology involves:

  • establishing a small number of targets that represent improved performance in key priority areas
  • regular reporting to keep the Premier and other ministers informed about progress towards targets and delivery risks
  • fieldwork to gather input from front‑line service staff and check on implementation of new or changed processes
  • analysing performance data to identify opportunities for performance improvement and to identify impediments to progress.

The division of responsibility between delivery units and the agencies they support differs between jurisdictions. In NSW, the targets were approved by Premier and Cabinet following discussions with agencies, prior to the establishment of the PIU. To support the agencies, the PIU:

  • develops performance reports in consultation with agencies
  • conducts fieldwork, usually in conjunction with agency staff
  • partners with agencies to analyse performance data.

In addition to these core functions, the PIU has trained agency staff responsible for implementation in how to adopt the methodology, and facilitates cross‑agency meetings so that key agency staff can share practice and experience.

3.2 Performance of the delivery unit

The PIU does not formally seek agency feedback

Agencies have no opportunity to formally comment on the value provided by the PIU, but most agencies we spoke to reported advantages of being associated with a Premier's Priority and working with the PIU such as:

  • central agency involvement that helps to break down organisational silos
  • scrutiny of performance data that helps to identify improvement opportunities
  • opportunities to share experience across government
  • being held to account
  • regular interaction with and commitment of the most senior levels of government.

Two agencies reported a level of discomfort about the close involvement of the PIU in reporting progress of their initiatives to the Premier. They feel unable to influence the messaging about the Priorities for which they are responsible.

Several agencies reported an interest in establishing a delivery function within their own agency to provide a similar level of support across selected programs and initiatives that are important to them.

The PIU approach aims to encourage agencies to take a broad view of who their stakeholders are

In line with the adopted methodology, the PIU supported some agencies to develop a ‘delivery chain’ which identified all of the organisations and individuals involved in delivering improved performance. Implementation plans were then developed to influence whichever parts of that delivery chain required improvement. Some delivery chains identify the need for multiple agencies to work together, even though achievement of the target is the responsibility of only one agency.

In some cases, agencies identify non‑government organisations or individual citizens as critical players within their delivery chain. Some agencies have adopted whole‑of‑government and community‑centric approaches to improving performance. An example is the Office of Sport collaborating with other agencies and with sporting organisations to provide incentives for children to increase their activity levels.

Exhibit 6: Office of Sport and other agencies are influencing both sporting organisations and families to increase children’s activity

The following diagram shows the delivery chain for active kids vouchers including that the leaders of the initiative are the Minister for Sport, Minister for Finance and Services and Minister for Health. The chain also shows that government operations, program and support staff, providers of services play an important role in the success of the initiative

Source: Department of Premier and Cabinet.

The PIU encourages a data‑driven approach

Some agencies commented upon the importance of data analysis that identified previously undetected performance improvement opportunities. Detailed analysis of granular performance data identified, among others:

  • specific schools with potential to move a large proportion of their NAPLAN results into the top two bands
  • a group of hospitals that required additional support to increase the number of patients through their emergency department in four hours
  • a small number of local councils with the potential to determine a large proportion of their housing approvals within 40 days
  • the opportunity to intervene earlier with domestic violence offenders, to prevent re-offending.

The approach of selectively targeting individual or small groups of front line service staff is new to some agencies and differs from the traditional approach of issuing updated guidelines and procedures across all front‑line service units. This approach can only work if the delivery unit and agencies have access to granular data that can reveal impediments to performance at specific locations or for particular groups of citizens.

The PIU brings agencies together to work collaboratively on problems

Almost all agencies reported that the collaborative approach adopted by the PIU adds value. In particular, agencies commented about the ability of PIU staff to break down organisational silos and bring agencies together. The attention of the Premier and other senior ministers provides agencies with the necessary impetus to put resources into collaboratively solving problems that may be the responsibility of just one agency. The PIU acts as the Premier's representative, encouraging agencies to solve problems together and facilitating cross‑agency collaboration.

Exhibit 7: Example of agencies collaborating

Department of Education and Ministry of Health are working towards healthier school canteens

The Healthy School Canteen Strategy is an intervention under the Premier's Priority to reduce childhood overweight and obesity by five per cent by 2025. The Strategy aims to support student health by increasing access to healthy food and drinks and make healthy food the easy choice at school.

The key goal of the Strategy is to ensure that all NSW Government school canteens are providing more healthy choices by the end of 2019. As at May 2018, at least 94 government schools have achieved a healthy canteen, providing healthy canteen access to more than 32,000 students. Work is ongoing to assist the remaining government school canteens to become healthy canteens by the end of 2019.

Successful implementation of the Healthy School Canteen Strategy relies on strong collaboration between the Department of Education and NSW Health. The Premier's Implementation Unit has supported this collaboration using data, fieldwork insights and reporting routines to drive delivery of the strategy. Based on this cross‑government collaboration, a range of tools and local support services have been provided to schools to make it easier for their canteens to become healthy.

 

a picture of a parent and a child at a school canteen

Source: Department of Premier and Cabinet.

The PIU monitors and reports risks to delivery

The PIU works with agencies to identify delivery risks. Risks are reported to the Premier and key ministers. The PIU maintains risk registers for some, but not all, initiatives associated with the Premier’s Priorities.

The performance of the PIU is reviewed both internally and externally, but the reviews are not independent

Performance of the PIU is assessed every six months by the organisation that owns the adopted methodology. Performance reports are shared with the Premier and other ministers. We reviewed a performance report from the middle of 2017 which identified that the PIU was performing well compared to other delivery units. The organisation that conducts these reviews was involved in the establishment of the PIU and cannot be considered an independent reviewer.

In addition, the PIU conducts periodic self‑assessments. While we have seen evidence of self‑reflection, the PIU does not have any externally‑validated KPIs against which performance can be independently assessed. There is an opportunity to periodically request formal agency feedback and to use this feedback to drive process improvement.

 

Return to top
 

 

 

Appendices

Appendix one: Response from agency

Appendix two: Data sources

Appendix three: About the audit

Appendix four: Performance auditing

 

 

Return to top