Refine search

Reports

27 September 2018

Published

Members' Additional Entitlements 2017

Premier and Cabinet

Compliance

Internal controls and governance

Management and administration

Regulation

Service delivery

13 September 2018

Published

Progress and measurement of the Premier's Priorities

Premier and Cabinet

Compliance

Internal controls and governance

Management and administration

Project management

Risk

Service delivery

Shared services and collaboration

Workforce and capability

The Premier’s Implementation Unit uses a systematic approach to measuring and reporting progress towards the Premier’s Priorities performance targets, but public reporting needed to improve, according to a report released today by the Auditor-General of NSW, Margaret Crawford.

The Premier of New South Wales has established 12 Premier’s Priorities. These are key performance targets for government.

The 12 Premier's Priorities
150,000 new jobs by 2019	Reduce the volume of litter by 40 per cent by 2020
10 key projects in metro and regional areas to be delivered on time and on budget, and nearly 90 local infrastructure projects to be delivered on time	Increase the proportion of NSW students in the top two NAPLAN bands by eight per cent by 2019
Increase the proportion of women in senior leadership roles in the NSW Government sector from 33 to 50 per cent by 2025 and double the number of Aboriginal and Torres Strait Islander people in senior leadership roles in the NSW Government sector, from 57 to 114	Increase the proportion of young people who successfully move from Specialist Homelessness Services to long-term accommodation to more than 34 per cent by 2019
61,000 housing completions on average per year to 2021	Reduce the proportion of domestic violence perpetrators reoffending by 25 per cent by 2021
Improve customer satisfaction with key government services every year, this term of government to 2019	Decrease the percentage of children and young people re-reported at risk of significant harm by 15 per cent by 2020
81 per cent of patients through emergency departments within four hours by 2019	Reduce overweight and obesity rates of children by five percentage points by 2025

Source: Department of Premier and Cabinet, Premier’s Priorities website.

Each Premier’s Priority has a lead agency and minister responsible for achieving the performance target.

The Premier’s Implementation Unit (PIU) was established within the Department of Premier and Cabinet (DPC) in 2015. The PIU is a delivery unit that supports agencies to measure and monitor performance, make progress toward the Premier’s Priorities targets, and report progress to the Premier, key ministers and the public.

This audit assessed how effectively the NSW Government is progressing and reporting on the Premier's Priorities.

The Premier’s Implementation Unit (PIU) is effective in assisting agencies to make progress against the Premier’s Priorities targets. Progress reporting is regular but transparency to the public is weakened by the lack of information about specific measurement limitations and lack of clarity about the relationship of the targets to broader government objectives.The PIU promotes a systematic approach to measuring performance and reporting progress towards the Premier’s Priorities’ performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives.

The PIU provides a systematic approach to measuring performance and reporting progress towards the Premier's Priorities performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives. The data used to measure the Premier’s Priorities comes from a variety of government and external datasets, some of which have known limitations. These limitations are not revealed in public reporting, and only some are revealed in progress reported to the Premier and ministers. This limits the transparency of reporting.

The PIU assists agencies to avoid unintended outcomes that can arise from prioritising particular performance measures over other areas of activity. The PIU has adopted a collaborative approach to assisting agencies to analyse performance using data, and helping them work across organisational silos to achieve the Premier’s Priorities targets.

Data used to measure progress for some of the Premier’s Priorities has limitations which are not made clear when progress is reported. This reduces transparency about the reported progress. Public reporting would also be improved with additional information about the relationship between specific performance measures and broader government objectives.

The PIU is responsible for reporting progress to the Premier, key ministers and the public. Agencies provide performance data and some play a role in preparing progress reports for the Premier and ministers. For 11 of the Premier's Priorities, progress is reported against measurable and time-related performance targets. For the infrastructure priority, progress is reported against project milestones.

Progress of some Priorities is measured using data that has known limitations, which should be noted wherever progress is reported. For example, the data used to report on housing completions does not take housing demolitions into account, and is therefore overstating the contribution of this performance measure to housing supply. This known limitation is not explained in progress reports or on the public website.

Data used to measure progress is sourced from a mix of government and external datasets. Updated progress data for most Premier’s Priorities is published on the Premier’s Priorities website annually, although reported to the Premier and key ministers more frequently. The PIU reviews the data and validates it through fieldwork with front line agencies. The PIU also assists agencies to avoid unintended outcomes that can arise from prioritising single performance measures. Most, but not all, agencies use additional indicators to check for misuse of data or perverse outcomes.

We examined the reporting processes and controls for five of the Premier’s Priorities. We found that there is insufficient assurance over the accuracy of the data on housing approvals.

The relationships between performance measures and broader government objectives is not always clearly explained on the Premier’s Priority website, which is the key source of public information about the Premier’s Priorities. For example, the Premier’s Priority to reduce litter volumes is communicated as “Keeping our Environment Clean.” While the website explains why reducing litter is important, it does not clearly explain why that particular target has been chosen to measure progress in keeping the environment clean.

By December 2018, the Department of Premier and Cabinet should:

improve transparency of public reporting by:
- providing information about limitations of reported data and associated performance
- clarifying the relationship between the Premier’s Priorities performance targets and broader government objectives.
ensure that processes to check and verify data are in place for all agency data sources
encourage agencies to develop and implement additional supporting indicators for all Premier’s Priority performance measures to prevent and detect unintended consequences or misuse of data.

The Premier's Implementation Unit is effective in supporting agencies to deliver progress towards the Premier’s Priority targets.

The PIU promotes a systematic approach to monitoring and reporting progress against a target, based on a methodology used in delivery units elsewhere in the world. The PIU undertakes internal self-evaluation, and commissions regular reviews of methodology implementation from the consultancy that owns the methodology and helped to establish the PIU. However, the unit lacks periodic independent reviews of their overall effectiveness. The PIU has adopted a collaborative approach and assists agencies to analyse performance using data, and work across organisational silos to achieve the Premier’s Priorities targets.

Agency representatives recognise the benefits of being responsible for a Premier's Priority and speak of the value of being held to account and having the attention of the Premier and senior ministers.

By June 2019, the Department of Premier and Cabinet should:

establish routine collection of feedback about PIU performance including:
- independent assurance of PIU performance
- opportunity for agencies to provide confidential feedback.

Appendix one: Response from agency

Appendix two: Data sources

Appendix three: About the audit

Appendix four: Performance auditing

Parliamentary reference - Report number #307 - released 13 September 2018

10 August 2018

Published

Managing Antisocial behaviour in public housing

Community Services

Asset valuation

Infrastructure

Regulation

Service delivery

Workforce and capability

The Department of Family and Community Services (FACS) has not adequately supported or resourced its staff to manage antisocial behaviour in public housing according to a report released today by the Deputy Auditor-General for New South Wales, Ian Goodwin.

In recent decades, policy makers and legislators in Australian states and territories have developed and implemented initiatives to manage antisocial behaviour in public housing environments. All jurisdictions now have some form of legislation or policy to encourage public housing tenants to comply with rules and obligations of ‘good neighbourliness’. In November 2015, the NSW Parliament changed legislation to introduce a new approach to manage antisocial behaviour in public housing. This approach is commonly described as the ‘strikes’ approach.

When introduced in the NSW Parliament, the ‘strikes’ approach was described as a means to:

improve the behaviour of a minority of tenants engaging in antisocial behaviour
create better, safer communities for law abiding tenants, including those who are ageing and vulnerable.

FACS has a number of tasks as a landlord, including a responsibility to collect rent and organise housing maintenance. FACS also has a role to support tenants with complex needs and manage antisocial behaviour. These roles have some inherent tensions. The FACS antisocial behaviour management policy aims are:

to balance the responsibilities of tenants, the rights of their neighbours in social housing, private residents and the broader community with the need to support tenants to sustain their public housing tenancies.

This audit assessed the efficiency and effectiveness of the ‘strikes’ approach to managing antisocial behaviour in public housing environments.

We examined whether:

the approach is being implemented as intended and leading to improved safety and security in social housing environments
FACS and its partner agencies have the capability and capacity to implement the approach
there are effective mechanisms to monitor, report and progressively improve the approach.

Conclusion

FACS has not adequately supported or resourced its staff to implement the antisocial behaviour policy. FACS antisocial behaviour data is incomplete and unreliable. Accordingly, there is insufficient data to determine the nature and extent of the problem and whether the implementation of the policy is leading to improved safety and security.

FACS management of minor and moderate incidents of antisocial behaviour is poor. FACS has not dedicated sufficient training to equip frontline housing staff with the relevant skills to apply the antisocial behaviour management policy. At more than half of the housing offices we visited, staff had not been trained to:

conduct effective interviews to determine whether an antisocial behaviour complaint can be substantiated
de escalate conflict and manage complex behaviours when required
properly manage the safety of staff and tenants
establish information sharing arrangements with police
collect evidence that meets requirements at the NSW Civil and Administrative Tribunal
record and manage antisocial behaviour incidents using the information management system HOMES ASB.

When frontline housing staff are informed about serious and severe illegal antisocial behaviour incidents, they generally refer them to the FACS Legal Division. Staff in the Legal Division are trained and proficient in managing antisocial behaviour in compliance with the policy and therefore, the more serious incidents are managed effectively using HOMES ASB.

FACS provides housing services to most remote townships via outreach visits from the Dubbo office. In remote townships, the policy is not being fully implemented due to insufficient frontline housing staff. There is very limited knowledge of the policy in these areas and FACS data shows few recorded antisocial behaviour incidents in remote regions.

The FACS information management system (HOMES ASB) is poorly designed and has significant functional limitations that impede the ability of staff to record and manage antisocial behaviour. Staff at most of the housing offices we visited were unable to accurately record antisocial behaviour matters in HOMES ASB, making the data incorrect and unreliable.

Parliamentary reference - Report number #306 - released 10 August 2018

29 June 2018

Published

Performance audit insights: key findings from 2014-2018

Whole of Government

Compliance

Fraud

Information technology

Internal controls and governance

Procurement

Project management

A report released today by the Auditor-General for New South Wales, Margaret Crawford, presents key findings from four years of performance audits. The report findings are presented around six areas of government activity including planning for the future, meeting community expectations for key services, investment in infrastructure, managing natural resources, ensuring good governance and digital disruption.

In this report, we present common findings and lessons from the past four years of performance audits, and offer insights to the public sector on elements of effective performance. We have analysed the key findings and recommendations from 61 performance audits tabled in the NSW Parliament between July 2014 and June 2018, spanning varied areas of government activity. We will also use this report to help determine areas of unaddressed risk across all parts of government, and to shape our future audit priorities.

Governments play an important stewardship role. Their decisions need to consider intergenerational equity by ensuring that investment strategies are sustainable. Governments also need to consider the impact of their decisions on different parts of the community. We recognise that governments face challenges in delivering programs and services, targeting complex social issues with finite resources.

Governments are changing how they deliver services to respond to citizen needs and deliver greater value for money. In this section, we reflect on audits that looked at how government entities are planning their activities to meet the needs of the community into the future.

State and local government exist to provide services to citizens, and citizens are playing a greater role in defining what services they want or need. Expectations about consultation, ease of access, timeliness, and customisation of services are rising. Governments face challenges to continually improve the way they plan and deliver services to meet these expectations. Governments also need to provide quality services for a growing and ageing population whilst working within a constrained financial environment.

Over the past four years, our performance audits have assessed aspects of State and local government services, including education, health services, disability support, corrective services, and many others. In this section, we draw together common findings that government entities should reflect on when providing services to the community.

The NSW Government’s 2018–19 Budget forecasts an $87.2 billion infrastructure investment program over the next four years. Infrastructure investment of this size carries significant opportunities and risks. Competition for resources is high and maintaining the capability to manage and deliver projects effectively is challenging. Governments also need to plan effectively to ensure infrastructure built today will meet future needs.

Over the past four years, we have looked at some of the ways NSW Government agencies justify and prioritise projects for funding, work with contractors to deliver projects, and track and report on progress. In this section, we draw together common findings from our audits that government entities should consider when planning future infrastructure projects.

Governments face challenges in balancing the use of natural resources to meet diverse interests, while supporting a sustainable natural environment for the future. They need to supply communities with water, produce energy, protect natural habitats, and support farming, industry, and economic development.

Some of our recent audits have considered how government agencies are managing natural resources and protecting the environment for future generations. In this section, we have drawn together common findings across our audits that government entities should consider in managing the environment and natural resources.

A range of checks and balances is needed to support public confidence in government decision making. To maintain trust, government agencies should act transparently, and in accordance with relevant legislation and policy. This is particularly important as the public sector increasingly engages with external partners to deliver services and provide a more contestable environment.

Good governance arrangements should result in improved service delivery and more effective and efficient use of resources. Our audits have looked at many different elements of governance, including making sure the necessary processes and workplace cultures are in place to help government entities achieve their aims. In this section, we have drawn together various aspects of governance that government entities should consider.

The global increase in digital technology provides governments with opportunities to interact with citizens in more immediate and responsive ways than was previously possible. Data can be used in powerful ways such as predicting future demand for services, targeting interventions, responding to crises, and evaluating outcomes. Governments face challenges in doing this while maintaining secure digital environments that protect citizen interests, privacy, and autonomy.

Our audits have assessed some of the ways that government entities are incorporating digital change into their work. In this section, we draw together common themes that governments could consider in protecting their digital assets, or expanding their digital capabilities.

Appendix one – Performance audits 2014 to 2018

Appendix two – Performance auditing

Appendix three – About this report

28 June 2018

Published

Regulation of water pollution in drinking water catchments and illegal disposal of solid waste

Environment

Compliance

Internal controls and governance

Management and administration

Regulation

Risk

There are important gaps in how the Environmental Protection Authority (EPA) implements its regulatory framework for water pollution in drinking water catchments and illegal solid waste disposal. This limits the effectiveness of its regulatory responses, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford.

The NSW Environment Protection Authority (the EPA) is the State’s primary environmental regulator. The EPA regulates waste and water pollution under the Protection of the Environment Operations Act 1997 (the Act) through its licensing, monitoring, regulation and enforcement activities. The community should be able to rely on the effectiveness of this regulation to protect the environment and human health. The EPA has regulatory responsibility for more significant and specific activities which can potentially harm the environment.

Activities regulated by the EPA include manufacturing, chemical production, electricity generation, mining, waste management, livestock processing, mineral processing, sewerage treatment, and road construction. For these activities, the operator must have an EPA issued environment protection licence (licence). Licences have conditions attached which may limit the amount and concentrations of substances the activity may produce and discharge into the environment. Conditions also require the licensee to report on its licensed activities.

This audit assessed the effectiveness of the EPA’s regulatory response to water pollution in drinking water catchments and illegal solid waste disposal. The findings and recommendations of this review can be reasonably applied to the EPA’s other regulatory functions, as the areas we examined were indicative of how the EPA regulates all pollution types and incidents.

Conclusion

There are important gaps in how the EPA implements its regulatory framework for water pollution in drinking water catchments and illegal solid waste disposal which limit the effectiveness of its regulatory response. The EPA uses a risk-based regulatory framework that has elements consistent with the NSW Government Guidance for regulators to implement outcomes and risk-based regulation. However, the EPA did not demonstrate that it has established reliable practices to accurately and consistently detect the risk of non compliances by licensees, and apply consistent regulatory actions. This may expose the risk of harm to the environment and human health.

The EPA also could not demonstrate that it has effective governance and oversight of its regulatory operations. The EPA operates in a complex regulatory environment where its regional offices have broad discretions for how they operate. The EPA has not balanced this devolved structure with an effective governance approach that includes appropriate internal controls to monitor the consistency or quality of its regulatory activities. It also does not have an effective performance framework that sets relevant performance expectations and outcome-based key performance indicators (KPIs) for its regional offices.

These deficiencies mean that the EPA cannot be confident that it conducts compliance and enforcement activities consistently across the State and that licensees are complying with their licence conditions or the Act.

The EPA's reporting on environmental and regulatory outcomes is limited and most of the data it uses is self reported by industry. It has not set outcome-based key result areas to assess performance and trends over time.

The EPA uses a risk-based regulatory framework for water pollution and illegal solid waste disposal but there are important gaps in implementation that reduce its effectiveness.

Elements of the EPA’s risk-based regulatory framework for water pollution and illegal solid waste disposal are consistent with the NSW Government Guidance for regulators to implement outcomes and risk-based regulation. There are important gaps in how the EPA implements its risk-based approach that limit the effectiveness of its regulatory response. The EPA could not demonstrate that it effectively regulates licensees because it has not established reliable practices that accurately and consistently detect licence non compliances or breaches of the Act and enforce regulatory actions.

The EPA lacks effective governance arrangements to support its devolved regional structure. The EPA's performance framework has limited and inconclusive reporting on regional performance to the EPA’s Chief Executive Officer or to the EPA Board. The EPA cannot assure that it is conducting its regulatory responsibilities effectively and efficiently.

The EPA does not consistently evaluate its regulatory approach to ensure it is effective and efficient. For example, there are no set requirements for how EPA officers conduct mandatory site inspections, which means that there is a risk that officers are not detecting all breaches or non-compliances. The inconsistent approach also means that the EPA cannot rely on the data it collects from these site inspections to understand whether its regulatory response is effective and efficient. In addition, where the EPA identifies instances of non compliance or breaches, it does not apply all available regulatory actions to encourage compliance.

The EPA also does not have a systematic approach to validate self-reported information in licensees’ annual returns, despite the data being used to assess administrative fees payable to the EPA and its regulatory response to non-compliances.

The EPA does not use performance frameworks to monitor the consistency or quality of work conducted across the State. The EPA has also failed to provide effective guidance for its staff. Many of its policies and procedures are out-dated, inconsistent, hard to access, or not mandated.

Recommendations

By 31 December 2018, to improve governance and oversight, the EPA should:
1.	implement a more effective performance framework with regular reports to the Chief Executive Officer and to the EPA Board on outcomes-based key result areas that assess its environmental and regulatory performance and trends over time
By 30 June 2019, to improve consistency in its practices, the EPA should:
2.	progressively update and make accessible its policies and procedures for regulatory operations, and mandate procedures where necessary to ensure consistent application
3.	implement internal controls to monitor the consistency and quality of its regulatory operations.

The EPA does not apply a consistent approach to setting licence conditions for discharges to water.

The requirements for setting licence conditions for water pollution are complex and require technical and scientific expertise. In August 2016, the EPA approved guidance developed by its technical experts in the Water Technical Advisory Unit to assist its regional staff. However, the EPA did not mandate the use of the guidance until mid-April 2018. Up until then, the EPA had left discretion to regional offices to decide what guidance their staff use. This meant that practices have differed across the organisation. The EPA is yet to conduct training for staff to ensure they consistently apply the 2016 guidance.

The EPA has not implemented any appropriate internal controls or quality assurance process to monitor the consistency or quality of licence conditions set by its officers across the State. This is not consistent with good regulatory practice.

The triennial 2016 audit of the Sydney drinking water catchment report highlighted that Lake Burragorang has experienced worsening water quality over the past 20 years from increased salinity levels. The salinity levels were nearly twice as high as in other storages in the Sydney drinking water catchment. The report recommended that the source and implication of the increased salinity levels be investigated. The report did not propose which public authority should carry out such an investigation.

To date, no NSW Government agency has addressed the report's recommendation. There are three public authorities, the EPA, DPE and WaterNSW that are responsible for regulating activities that impact on water quality in the Sydney drinking water catchment, which includes Lake Burragorang.

Recommendation

By 30 June 2019, to address worsening water quality in Lake Burragorang, the EPA should:
4.	(a) review the impact of its licensed activities on water quality in Lake Burragorang, and
	(b) develop strategies relating to its licensed activities (in consultation with other relevant NSW Government agencies) to improve and maintain the lake's water quality.

The EPA’s risk-based approach to monitoring compliance of licensees has limited effectiveness.

The EPA tailors its compliance monitoring approach based on the performance of licensees. This means that licensees that perform better have a lower administrative fee and fewer mandatory site inspections.

However, this approach relies on information that is not complete or accurate. Sources of information include licensees’ annual returns, EPA site inspections and compliance audits, and pollution reports from the public.

Licensees report annually to the EPA on their performance, including compliance against their licence conditions. The Act contains significant financial penalties if licensees provide false and misleading information in their annual returns. However, the EPA does not systematically or consistently validate information self-reported by licensees, or consistently apply regulatory actions if it discovers non-compliance.

Self-reported compliance data is used in part to assess a licensed premises’ overall environmental risk level, which underpins the calculation of the administrative fee, the EPA’s site inspection frequency, and the licensee’s exposure to regulatory actions. It is also used to assess the load-based licence fee that the licensee pays.

The EPA has set minimum mandatory site inspection frequencies for licensed premises based on its assessed overall risk level. This is a key tool to detect non-compliance or breaches of the Act. However, the EPA has not issued a policy or procedures that define what these mandatory inspections should cover and how they are to be conducted. We found variations in how the EPA officers in the offices we visited conducted these inspections. The inconsistent approach means that the EPA does not have complete and accurate information of licensees’ compliance. The inconsistent approach also means that the EPA is not effectively identifying all non-compliances for it to consider applying appropriate regulatory actions.

The EPA also receives reports of pollution incidents from the public that may indicate non-compliance. However, the EPA has not set expected time frames within which it expects its officers to investigate pollution incidents. The EPA regional offices decide what to investigate and timeframes. The EPA does not measure regional performance regarding timeframes.

The few compliance audits the EPA conducts annually are effective in identifying licence non-compliances and breaches of the Act. However, the EPA does not have a policy or required procedures for its regulatory officers to consistently apply appropriate regulatory actions in response to compliance audit findings.

The EPA has not implemented any effective internal controls or quality assurance process to check the consistency or quality of how its regulatory officers monitor compliance across the State. This is not consistent with good regulatory practice.

Recommendations

To improve compliance monitoring, the EPA should implement procedures to:
5.	by 30 June 2019, validate self-reported information, eliminate hardcopy submissions and require licensees to report on their breaches of the Act and associated regulations in their annual returns
6.	by 31 December 2018, conduct mandatory site inspections under the risk-based licensing scheme to assess compliance with all regulatory requirements and licence conditions.

The EPA cannot assure that its regulatory enforcement approach is fully effective.

The EPA’s compliance policy and prosecution guidelines have a large number of available regulatory actions and factors which should be taken into account when selecting an appropriate regulatory response. The extensive legislation determining the EPA’s regulatory activities, and the devolved regional structure the EPA has adopted in delivering its compliance and regulatory functions, increases the risk of inconsistent compliance decisions and regulatory responses. A good regulatory framework needs a consistent approach to enforcement to incentivise compliance.

The EPA has not balanced this devolved regional structure with appropriate governance arrangements to give it assurance that its regulatory officers apply a consistent approach to enforcement.

The EPA has not issued standard procedures to ensure consistent non-court enforcement action for breaches of the Act or non-compliance with licence conditions. Given our finding that the EPA does not effectively detect breaches and non-compliances, there is a risk that it is not applying appropriate regulatory actions for many breaches and non-compliances.

A recent EPA compliance audit identified significant non-compliances with incident management plan requirements. However, the EPA has not applied regulatory actions for making false statements on annual returns for those licensees that certified their plans complied with such requirements. The EPA also has not applied available regulatory actions for the non-compliances which led to the false or misleading statements.

Recommendation

By 31 December 2018 to improve enforcement, the EPA should:
7.	Implement procedures to systematically assess non-compliances with licence conditions and breaches of the Act and to implement appropriate and consistent regulatory actions.

The EPA has implemented the actions listed in the NSW Illegal Dumping Strategy 2014–16. To date, the EPA has also implemented four of the six recommendations made by the ICAC on EPA's oversight of Regional Illegal Dumping Squads.

The EPA did not achieve the NSW Illegal Dumping Strategy 2014–16 target of a 30 per cent reduction in instances of large scale illegal dumping in Sydney, the Illawarra, Hunter and Central Coast from 2011 levels.

In the reporting period, the incidences of large scale illegal dumping more than doubled. The EPA advised that this increase may be the result of greater public awareness and reporting rather than increased illegal dumping activity.

By June 2018, the EPA is due to implement one outstanding recommendation made by the ICAC but has not set a time for the other outstanding recommendation.

Appendix one – Response from agency

Appendix two – List of enforcement tools

Appendix three – The EPA's organisational structure

Appendix four – The EPA's regions and branches

Appendix five – About the audit

Appendix six – Performance auditing

Parliamentary reference - Report number #304 - released 28 June 2018

7 June 2018

Published

HealthRoster benefits realisation

Health

Compliance

Information technology

Management and administration

Project management

Workforce and capability

The HealthRoster system is delivering some business benefits but Local Health Districts are yet to use all of its features, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford. HealthRoster is an IT system designed to more effectively roster staff to meet the needs of Local Health Districts and other NSW health agencies.

The NSW public health system employs over 100,000 people in clinical and non-clinical roles across the state. With increasing demand for services, it is vital that NSW Health effectively rosters staff to ensure high quality and efficient patient care, while maintaining good workplace practices to support staff in demanding roles.

NSW Health is implementing HealthRoster as its single state-wide rostering system to more effectively roster staff according to the demands of each location. Between 2013–14 and 2016–17, our financial audits of individual LHDs had reported issues with rostering and payroll processes and systems.

NSW Health grouped all Local Health Districts (LHDs), and other NSW Health organisations, into four clusters to manage the implementation of HealthRoster over four years. Refer to Exhibit 4 for a list of the NSW Health entities in each cluster.

Cluster 1 implementation commenced in 2014–15 and was completed in 2015–16.
Cluster 2 implementation commenced in 2015–16 and was completed in 2016–17.
Cluster 3 began implementation in 2016–17 and was underway during the conduct of the audit.
Cluster 4 began planning for implementation in 2017–18.

Full implementation, including capability for centralised data and reporting, is planned for completion in 2019.

This audit assessed the effectiveness of the HealthRoster system in delivering business benefits. In making this assessment, we examined whether:

expected business benefits of HealthRoster were well-defined
HealthRoster is achieving business benefits where implemented.

The HealthRoster project has a timespan from 2009 to 2019. We examined the HealthRoster implementation in LHDs, and other NSW Health organisations, focusing on the period from 2014, when eHealth assumed responsibility for project implementation, to early 2018.

Conclusion

The HealthRoster system is realising functional business benefits in the LHDs where it has been implemented. In these LHDs, financial control of payroll expenditure and rostering compliance with employment award conditions has improved. However, these LHDs are not measuring the value of broader benefits such as better management of staff leave and overtime.

NSW Health has addressed the lessons learned from earlier implementations to improve later implementations. Business benefits identified in the business case were well defined and are consistent with business needs identified by NSW Health. Three of four cluster 1 LHDs have been able to reduce the number of issues with rostering and payroll processes. LHDs in earlier implementations need to use HealthRoster more effectively to ensure they are getting all available benefits from it.

HealthRoster is taking six years longer, and costing $37.2 million more, to fully implement than originally planned. NSW Health attributes the increased cost and extended timeframe to the large scale and complexity of the full implementation of HealthRoster.

Business benefits identified for HealthRoster accurately reflect business needs.

NSW Health has a good understanding of the issues in previous rostering systems and has designed HealthRoster to adequately address these issues. Interviews with frontline staff indicate that HealthRoster facilitates rostering which complies with industrial awards. This is a key business benefit that supports the provision of quality patient care. We saw no evidence that any major business needs or issues with the previous rostering systems are not being addressed by HealthRoster.

In the period examined in this audit since 2015, NSW Health has applied appropriate project management and governance structures to ensure that risks and issues are well managed during HealthRoster implementation.

HealthRoster has had two changes to its budget and timeline. Overall, the capital cost for the project has increased from $88.6 million to $125.6 million (42 per cent) and has delayed expected project completion by four years from 2015 to 2019. NSW Health attributes the increased cost and extended time frame to the large scale and complexity of the full implementation of HealthRoster.

NSW Health has established appropriate governance arrangements to ensure that HealthRoster is successfully implemented and that it will achieve business benefits in the long term. During implementation, local steering committees monitor risks and resolve implementation issues. Risks or issues that cannot be resolved locally are escalated to the state-wide steering committee.

NSW Health has grouped local health districts, and other NSW Health organisations, into four clusters for implementation. This has enabled NSW Health to apply lessons learnt from each implementation to improve future implementations.

NSW Health has a benefits realisation framework, but it is not fully applied to HealthRoster.

NSW Health can demonstrate that HealthRoster has delivered some functional business benefits, including rosters that comply with a wide variety of employment awards.

NSW Health is not yet measuring and tracking the value of business benefits achieved. NSW Health did not have benefits realisation plans with baseline measures defined for LHDs in cluster 1 and 2 before implementation. Without baseline measures NSW Health is unable to quantify business benefits achieved. However, analysis of post-implementation reviews and interviews with frontline staff indicate that benefits are being achieved. As a result, NSW Health now includes defining baseline measures and setting targets as part of LHD implementation planning. It has created a benefits realisation toolkit to assist this process from cluster 3 implementations onwards.

NSW Health conducted post-implementation reviews for clusters 1 and 2 and found that LHDs in these clusters were not using HealthRoster to realise all the benefits that HealthRoster could deliver.

By September 2018, NSW Health should:

Ensure that Local Health Districts undertake benefits realisation planning according to the NSW Health benefits realisation framework
Regularly measure benefits realised, at state and local health district levels, from the statewide implementation of HealthRoster
Review the use of HealthRoster in Local Health Districts in clusters 1 and 2 and assist them to improve their HealthRoster related processes and practices.

By June 2019, NSW Health should:

Ensure that all Local Health Districts are effectively using demand based rostering.

Appendix one - Response from agency

Appendix two - About the audit

Appendix three - Performance auditing

Parliamentary reference - Report number #301 - released 7 June 2018

17 May 2018

Published

Regional Assistance Programs

Premier and Cabinet

Planning

Transport

Compliance

Infrastructure

Management and administration

Project management

Infrastructure NSW effectively manages how grant applications for regional assistance programs are assessed and recommended for funding. Its contract management processes are also effective. However, we are unable to conclude whether the objectives of these programs have been achieved as the relevant agencies have not yet measured their benefits, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford.

In 2011, the NSW Government established Restart NSW to fund new infrastructure with the proceeds from the sale and lease of government assets. From 2011 to 2017, the NSW Government allocated $1.7 billion from the fund for infrastructure in regional areas, with an additional commitment of $1.3 billion to be allocated by 2021. The NSW Government allocates these funds through regional assistance programs such as Resources for Regions and Fixing Country Roads. NSW councils are the primary recipients of funding provided under these programs.

The NSW Government announced the Resources for Regions program in 2012 with the aim of addressing infrastructure constraints in mining affected communities. Infrastructure NSW administers the program, with support from the Department of Premier and Cabinet.

The NSW Government announced the Fixing Country Roads program in 2014 with the aim of building more efficient road freight networks. Transport for NSW and Infrastructure NSW jointly administer this program, which funds local councils to deliver projects that help connect local and regional roads to state highways and freight hubs.

This audit assessed whether these two programs (Resources for Regions and Fixing Country Roads) were being effectively managed and achieved their objectives. In making this assessment, we answered the following questions:

How well are the relevant agencies managing the assessment and recommendation process?
How do the relevant agencies ensure that funded projects are being delivered?
Do the funded projects meet program and project objectives?

The audit focussed on four rounds of Resources for Regions funding between 2013–14 to 2015–16, as well as the first two rounds of Fixing Country Roads funding in 2014–15 and 2015–16.

Conclusion

Infrastructure NSW effectively manages how grant applications are assessed and recommended for funding. Infrastructure NSW’s contract management processes are also effective. However, we are unable to conclude on whether program objectives are being achieved as Infrastructure NSW has not yet measured program benefits.

While Infrastructure NSW and Transport for NSW managed the assessment processes effectively overall, they have not fully maintained all required documentation, such as conflict of interest registers. Keeping accurate records is important to support transparency and accountability to the public about funding allocation. The relevant agencies have taken steps to address this in the current funding rounds for both programs.

For both programs assessed, the relevant agencies have developed good strategies over time to support councils through the application process. These strategies include workshops, briefings and feedback for unsuccessful applicants. Transport for NSW and the Department of Premier and Cabinet have implemented effective tools to assist applicants in demonstrating the economic impact of their projects.

Infrastructure NSW is effective in identifying projects that are 'at‑risk' and assists in bringing them back on track. Infrastructure NSW has a risk‑based methodology to verify payment claims, which includes elements of good practice in grants administration. For example, it requires grant recipients to provide photos and engages Public Works Advisory to review progress claims and visit project sites.

Infrastructure NSW collects project completion reports for all Resources for Regions and Fixing Country Roads funded projects. Infrastructure NSW intends to assess benefits for both programs once each project in a funding round is completed. To date, no funding round has been completed. As a result, no benefits assessment has been done for any completed project funded in either program.

The project selection criteria are consistent with the program objectives set by the NSW Government, and the RIAP applied the criteria consistently. Probity and record keeping practices did not fully comply with the probity plans.

The assessment methodology designed by Infrastructure NSW is consistent with2 the program objectives and criteria. In the rounds that we reviewed, all funded projects met the assessment criteria.

Infrastructure NSW developed probity plans for both programs which provided guidance on the record keeping required to maintain an audit trail, including the use of conflict of interest registers. Infrastructure NSW and Transport for NSW did not fully comply with these requirements. The relevant agencies have taken steps to address this in the current funding rounds for both programs.

NSW Procurement Board Directions require agencies to ensure that they do not engage a probity advisor that is engaged elsewhere in the agency. Infrastructure NSW has not fully complied with this requirement. A conflict of interest arose when Infrastructure NSW engaged the same consultancy to act as its internal auditor and probity advisor.

While these infringements of probity arrangements are unlikely to have had a major impact on the assessment process, they weaken the transparency and accountability of the process.

Some councils have identified resourcing and capability issues which impact on their ability to participate in the application process. For both programs, the relevant agencies conducted briefings and webinars with applicants to provide advice on the objectives of the programs and how to improve the quality of their applications. Additionally, Transport for NSW and the Department of Premier and Cabinet have developed tools to assist councils to demonstrate the economic impact of their applications.

The relevant agencies provided feedback on unsuccessful applications to councils. Councils reported that the quality of this feedback has improved over time.

Recommendations

By June 2018, Infrastructure NSW should:
- ensure probity reports address whether all elements of the probity plan have been effectively implemented.

By June 2018, Infrastructure NSW and Transport for NSW should:
- maintain and store all documentation regarding assessment and probity matters according to the State Records Act 1998, the NSW Standard on Records Management and the relevant probity plans

Infrastructure NSW is responsible for overseeing and monitoring projects funded under Resources for Regions and Fixing Country Roads. Infrastructure NSW effectively manages projects to keep them on track, however it could do more to assure itself that all recipients have complied with funding deeds. Benefits and outcomes should also start to be measured and reported as soon as practicable after projects are completed to inform assessment of future projects.

Infrastructure NSW identifies projects experiencing unreasonable delays or higher than expected expenses as 'at‑risk'. After Infrastructure NSW identifies a project as 'at‑risk', it puts in place processes to resolve issues to bring them back on track. Infrastructure NSW, working with Public Works Advisory regional offices, employs a risk‑based approach to validate payment claims, however this process should be strengthened. Infrastructure NSW would get better assurance by also conducting annual audits of compliance with the funding deed for a random sample of projects.

Infrastructure NSW collects project completion reports for all Resources for Regions and Fixing Country Roads funded projects. It applies the Infrastructure Investor Assurance Framework to Resources for Regions and Fixing Country Roads at a program level. This means that each round of funding (under both programs) is treated as a distinct program for the purposes of benefits realisation. It plans to assess whether benefits have been realised once each project in a funding round is completed. As a result, no benefits realisation assessment has been done for any project funded under either Resources for Regions or Fixing Country Roads. Without project‑level benefits realisation, future decisions are not informed by the lessons from previous investments.

Recommendations

By December 2018, Infrastructure NSW should:
- conduct annual audits of compliance with the funding deed for a random sample of projects funded under Resources for Regions and Fixing Country Roads
- publish the circumstances under which unspent funds can be allocated to changes in project scope
- measure benefits delivered by projects that were completed before December 2017
- implement an annual process to measure benefits for projects completed after December 2017

By December 2018, Transport for NSW and Infrastructure NSW should:
- incorporate a benefits realisation framework as part of the detailed application.

Appendix one - Response from agencies

Appendix two - Maps of funded projects

Appendix three - About the audit

Appendix four - Performance auditing

Parliamentary reference - Report number #300 - released 17 May 2018

24 September 2008

Published

Delivering Health Care out of Hospitals

Health

Management and administration

Project management

Service delivery

Workforce and capability

Area Health Services and hospitals have developed programs which can provide clinical outcomes as good for patients as in-hospital care and can reduce the time they spend in hospital. They have operated for several years and show considerable potential. Ageing of the population, increasing costs and higher expectations of health care will continue to challenge health systems. Sometimes, unfortunately, the system struggles to cope.

Parliamentary reference - Report number #182 - released 24 September 2008

21 May 2008

Published

Protecting our Rivers

Environment

Industry

Compliance

Internal controls and governance

Management and administration

Regulation

Shared services and collaboration

At the time of our 2003 audit agencies were unable to measure the water quality in NSW rivers as they had not established a comprehensive and ongoing monitoring system for water quality. Agencies are now developing a monitoring system to measure the progress against the State Plan 2006 target to improve the condition of riverine ecosystems by 2015. It may not be possible to identify an overall trend in the condition of riverine ecosystems by the State Plan target date of 2015. This is because riverine ecosystems do not respond rapidly to change in the environment. A long period of time is required to collect sufficient data to establish a baseline for all indicators being used and to determine trends, particularly given the prolonged drought.

Parliamentary reference - Report number #180 - released 21 May 2008

23 April 2008

Published

Working with Hotels and Clubs to Reduce Alcohol-Related Crime

Justice

Compliance

Management and administration

Regulation

Service delivery

Shared services and collaboration

Workforce and capability

In NSW alcohol-related assaults have almost doubled over the last ten years. In 2006-07 there were 20,475 alcohol-related assaults excluding domestic violence incidents, up from 10,305 in 1997-98. In fact, the number of these assaults is increasing at a faster rate than total assaults. Incidences of alcohol-related malicious damage and offensive conduct have also increased, rising by 87 and 70 per cent respectively. Working with hotels and clubs to reduce alcohol-related crime, however we found that the number of alcohol-related assaults has reduced or stabilised in the last two years in about 14 per cent of Police local area commands.

Parliamentary reference - Report number #179 - released 23 April 2008