Executive summary

 The focus of our audit

Agencies need to define meaningful performance measures and develop performance information to manage services and to report to Government, Parliament and the public on how they ensure services are efficient and effective and provide value for money.

Sound financial and performance information is essential to establish priorities and inform decisions on whether to expand, contract, relocate or cease a service. Previous audits have found some agencies do not have sufficient information to successfully manage their programs and lack critical performance information needed to determine what to improve, by how much and how quickly.

In this performance audit, we examined ten different programs to see if this was generally the case. We looked at what information was collected and how this information was used to manage the program, improve services and achieve results.

Audit opinion

Overall the results were mixed. There is some good news but this is such a basic and vital issue that we must conclude that a good deal more needs to be done. Three agencies did not have sufficient information to provide a balanced view of services. And two of these agencies could not tell us whether their services actually made a difference to customers. Across the ten programs we found many examples of good practice, but some variation in the quality and coverage of performance measures.

Agencies that we identified as not having sufficient information to judge services were either unaware of its importance, collected data on activities but not results or reported system limitations.

Inadequate performance information presents a risk to effective management and overall accountability. And to date, despite central agency guidance and feedback to agencies, there is still considerable variation in the quality of performance information.

Agencies also need to set clear and concrete performance expectations or targets in order to track progress and use benchmarks to demonstrate achievements and best practice. Nine programs had established targets against which performance was compared, but agencies were particularly reluctant to publish the results.

Agencies also shied away from using performance information on similar programs in other states to compare results in NSW and to identify where services could be improved or savings made.

Recent reforms to the budget process that link agency funding to the delivery of services are an effective way of getting agencies to focus on the need for good performance measures and may lead to further improvements in the quality of performance information used by an agency to manage results.

Recommendations

It is recommended that NSW Treasury further assist agencies to improve the quality of performance information by:

§         providing additional guidance on developing performance measures consistent with the principles listed in exhibit 5 and in NSW Treasury’s own guidelines (page 20)

§         continuing to provide feedback to agencies on the quality and coverage of performance measures used in Results and Services Plans (page 20).

It is recommended that NSW Treasury require agencies to:

§         publish Results and Services Plans that include results logic (the link between services provided and their impact), performance measures and performance expectations (page 26)

§         report on results for these performance measures against performance expectations in annual reports (page 26).

It is recommended that NSW Treasury consolidate information on government priorities for Results and Services Plans and report publicly on whole-of-government outcomes (page 12).

Key audit findings

Chapter 1:
What should agencies use to measure

Agencies need sufficient information to judge if their services meet the expectations of government and the needs of citizens, and if improvements can be made.

performance?

The primary reasons for having a performance measurement system in place are to improve internal decision-making and to meet external accountability requirements. As a minimum, these systems should provide sufficient data to ensure that services are efficient and effective and provide value for money.

We found the requirement that general government sector agencies prepare a Results and Services Plan each year a useful basis for identifying what performance information each agency needs to manage services.

Most case study agencies had developed a Results and Services Plan. These plans present non-financial performance information in a consistent format, and each agency had also developed performance measures as part of this process.

These plans should reflect government expectations and priorities, although key government priorities are sometimes difficult to identify.

Linking funding decisions to the delivery of services is an effective way of getting agencies to focus on measuring performance. Whole-of-government plans would consolidate information on government priorities and whole-of-government reports would be useful for keeping Parliament and the public informed about results.

Chapter 2:
What performance information do agencies use?

In previous audits we reported that some agencies do not publish complete sets of performance information, making it difficult to judge service quality. However, we were unsure if this was because the data was not available or the agency had chosen not to publish the result.

Overall, we found that there was a lot more performance information being collected by managers to monitor programs, than what was appearing in published documents. Yet, we also found variation in the quality and coverage of performance information.

Chapter 3:
How do agencies use performance information?

Performance information should be used to monitor and plan services and judge results. Performance information should also be used to hold managers to account.

Eight of the programs effectively used performance information to redirect resources to areas of highest priority or to address changes in customer needs.

Agencies need to set clear and concrete performance expectations or targets in order to track progress and use benchmarks to demonstrate achievements.

Nine programs had established targets against which performance was compared, but agencies were particularly reluctant to use performance information from other states to benchmark results or identify best practice.

Response from NSW Treasury

Thank you for the invitation to comment on the Performance Audit report Agency Use of Performance Information to Manage Services.

Whilst the report observes variations in the quality and coverage of performance information in the case studies examined, Treasury is pleased to note the report’s focus on opportunities for improving information for managing and reporting service delivery performance.

It is also very encouraging that the report acknowledges the effectiveness of recent Government initiatives that both link agency strategic planning with the Budget process and promote the development by agencies of meaningful performance information.

In particular, the report finds that the Results and Services Plan, a high level service delivery and funding plan prepared by Ministers and agencies for the Budget Committee of Cabinet, is a ‘useful basis for identifying what performance information each agency needs to manage services’.

Treasury considers the Results and Services Plan approach fundamental to improving the quality of agency performance information and its use in program management and reporting. This is because the approach encourages agencies to apply a better practice method for:

·      explaining how their services are expected to contribute to results,

·      identifying which attributes of performance need to be measured, and

·      selecting meaningful measures of service accomplishment and indicators of results achievement.

Treasury notes that the report’s five recommendations propose consolidation of the Results and Services Plan process and are, in many cases, already being implemented.

Recommendations one and two propose that Treasury improve the quality of performance information by providing agencies with additional guidance material and ongoing feedback on the performance measures captured in Results and Services Plans.

Treasury currently provides agencies with a package of practical guidelines for developing and selecting useful performance information, including What You Do and Why: An Agency Guide to Defining Results and Services (TPP04-4).

These guidelines set a standard for performance measurement coverage that is consistent with, and in some respects exceeds, the better practice principles set out in the report. Nonetheless, Treasury is currently developing additional guidance material on performance measurement and expects that this will be issued as part of the 2007‑08 Budget cycle.

As part of ongoing agency relationship management, Treasury analysts routinely provide agencies with feedback on the quality and coverage of performance information captured in Results and Services Plans.

Recommendations three and four propose that Treasury require agencies to publish Results and Services Plans and to report both achieved and planned performance in annual reports.

Information on results and services has been reported in the commentary section of the 2006-07 Budget Estimates.  Based on the Results and Services Plan, this development will provide more detailed information on the services that agencies deliver and the results towards which they work.

In the February 2006 Economic and Financial Statement the Government committed to developing a new Performance Management and Budgeting System for implementation in the 2008-09 Budget.  The new system will build on the existing Results and Services Plan Budget process.

As the report notes, Results and Services Plans are Cabinet-in-Confidence documents and are therefore not for publication.  However, as part of the development process for the new Performance Management and Budgeting System the Government will consider how agency performance reporting can be simplified and aligned with elements of the Results and Services Plan framework.

Recommendation five proposes that Treasury consolidate information on government priorities for Results and Services Plans and report publicly on whole-of-government outcomes.

The first milestone in the development of the new Performance Management and Budgeting System will be the publication of the Government’s commitment to Serving the Public Better by the end of September 2006.

This statement will be the basis for priority setting within Government in subsequent budget years and will provide a framework for agreeing performance expectations in Results and Services Plans.

In this context, options for reporting whole-of-government priorities and performance will be considered as part of the development of the new Performance Management and Budgeting System.

I would like to thank the Audit Office for the cooperative manner in which the performance audit was conducted.

(signed)

Mark Ronsisvalle

Deputy Secretary

Dated: 13 June 2006