Conclusion
1. The urban renewal projects on the former railway land are well targeted to support the objectives of the Program. However, there is insufficient evidence that the cost of the light rail will be justified by its contribution to Program objectives.

The planned uses of the former railway land achieve a balance between the economic and social objectives of the Program at a reasonable cost to the Government. HCCDC, and previously UrbanGrowth NSW, identified and considered options for land use that would best meet Program objectives. Required probity processes were followed for developments that involved financial transactions. Our audit did not assess the achievement of these objectives because none of the projects have been completed yet.

Analysis presented in the Program business case and other planning documents showed that the light rail would have small transport benefits and was expected to make a modest contribution to broader Program objectives. Analysis in the Program business case argued that despite this, the light rail was justified because it would attract investment and promote economic development around the route. The Program business case referred to several international examples to support this argument, but did not make a convincing case that these examples were comparable to the proposed light rail in Newcastle.

The audited agencies argue that the contribution of light rail cannot be assessed separately because it is a part of a broader Program. The cost of the light rail makes up around 53 per cent of the total Program funding. Given the cost of the light rail, agencies need to be able to demonstrate that this investment provides value for money by making a measurable contribution to the Program objectives.

2. Consultation and oversight were mostly effective during the implementation stages of the Program. There were weaknesses in both areas in the planning stages.

Consultations about the urban renewal activities from around 2015 onward followed good practice standards. These consultations were based on an internationally accepted framework and met their stated objectives. Community consultations on the decision to close the train line were held in 2006 and 2009. However, the final decision in 2012 was made without a specific community consultation. There was no community consultation on the decision to build a light rail.

The governance arrangements that were in place during the planning stages of the Program did not provide effective oversight. This meant there was not a single agreed set of Program objectives until 2016 and roles and responsibilities for the Program were not clear. Leadership and oversight improved during the implementation phase of the Program. Roles and responsibilities were clarified and a multi-agency steering committee was established to resolve issues that needed multi-agency coordination.

Analysis presented in Program business cases and other planning documents showed that the light rail would have small transport benefits and was expected to make a modest contribution to broader Program objectives. Analysis in the Program business case argued that despite this, the light rail was justified because it would attract investment and promote economic development around the route. The Program business case referred to several international examples to support this argument, but did not make a convincing case that these examples were comparable to the proposed light rail in Newcastle.

The business case analysis of the benefits and costs of light rail was prepared after the decision to build light rail had been made and announced. Our previous reports, and recent reports by others, have emphasised the importance of completing thorough analysis before announcing infrastructure projects. Some advice provided after the initial light rail decision was announced was overly optimistic. It included benefits that cannot reasonably be attributed to light rail and underestimated the scope and cost of the project.

The audited agencies argue that the contribution of light rail cannot be assessed separately because it is part of a broader Program. The cost of the light rail makes up around 53 per cent of the total Program funding. Given the high cost of the light rail, we believe agencies need to be able to demonstrate that this investment provides value for money by making a measurable contribution to the Program objectives.

Recommendations
For future infrastructure programs, NSW Government agencies should support economical decision-making on infrastructure projects by:

• providing balanced advice to decision makers on the benefits and risks of large infrastructure investments at all stages of the decision-making process
• providing scope and cost estimates that are as accurate and complete as possible when initial funding decisions are being made
• making business cases available to the public.​​​​​​

The planned uses of the former railway land achieve a balance between the economic and social objectives of the Program at a reasonable cost to the government.

The planned uses of the former railway land align with the objectives of encouraging people to visit and live in the city centre, creating attractive public spaces, and supporting growth in employment in the city. The transport benefits of the activities are less clear, because the light rail is the major transport project and this will not make significant improvements to transport in Newcastle.

The processes used for selling and leasing parts of the former railway land followed industry standards. Options for the former railway land were identified and assessed systematically. Competitive processes were used for most transactions and the required assessment and approval processes were followed. The sale of land to the University of Newcastle did not use a competitive process, but required processes for direct negotiations were followed.

Recommendation
By March 2019, the Hunter and Central Coast Development Corporation should:

• work with relevant stakeholders to explore options for increasing the focus on the heritage objective of the Program in projects on the former railway land. This could include projects that recognise the cultural and industrial heritage of Newcastle.

Consultations about the urban renewal activities followed good practice standards, but consultation on transport decisions for the Program did not.

Consultations focusing on urban renewal options for the Program included a range of stakeholders and provided opportunities for input into decisions about the use of the former railway land. These consultations received mostly positive feedback from participants. Changes and additions were made to the objectives of the Program and specific projects in response to feedback received. 

There had been several decades of debate about the potential closure of the train line, including community consultations in 2006 and 2009. However, the final decision to close the train line was made and announced in 2012 without a specific community consultation. HCCDC states that consultation with industry and business representatives constitutes community consultation because industry representatives are also members of the community. This does not meet good practice standards because it is not a representative sample of the community.

There was no community consultation on the decision to build a light rail. There were subsequent opportunities for members of the community to comment on the implementation options, but the decision to build it had already been made. A community and industry consultation was held on which route the light rail should use, but the results of this were not made public. 

Recommendation
For future infrastructure programs, NSW Government agencies should consult with a wide range of stakeholders before major decisions are made and announced, and report publicly on the results and outcomes of consultations. 

The governance arrangements that were in place during the planning stages of the Program did not provide effective oversight. Project leadership and oversight improved during the implementation phase of the Program.

Multi-agency coordination and oversight were ineffective during the planning stages of the Program. Examples include: multiple versions of Program objectives being in circulation; unclear reporting lines for project management groups; and poor role definition for the initial advisory board. Program ownership was clarified in mid-2016 with the appointment of a new Program Director with clear accountability for the delivery of the Program. This was supported by the creation of a multi-agency steering committee that was more effective than previous oversight bodies.

The limitations that existed in multi-agency coordination and oversight had some negative consequences in important aspects of project management for the Program. This included whole-of-government benefits management and the coordination of work to mitigate impacts of the Program on small businesses.

Recommendations
For future infrastructure programs, NSW Government agencies should: 

• develop and implement a benefits management approach from the beginning of a program to ensure responsibility for defining benefits and measuring their achievement is clear
• establish whole-of-government oversight early in the program to guide major decisions. This should include:
  • agreeing on objectives and ensuring all agencies understand these
  • clearly defining roles and responsibilities for all agencies
  • establishing whole-of-government coordination for the assessment and mitigation of the impact of major construction projects on businesses and the community.

By March 2019, the Hunter and Central Coast Development Corporation should update and implement the Program Benefits Realisation Plan. This should include:

• setting measurable targets for the desired benefits
• clearly allocating ownership for achieving the desired benefits
• monitoring progress toward achieving the desired benefits and reporting publicly on the results.

Conclusion

The mobile speed camera program requires improvements to key aspects of its management to maximise road safety benefits. While camera locations have been selected based on crash history, the limited number of locations restricts network coverage. It also makes enforcement more predictable, reducing the ability to provide a general deterrence. Implementation of the program has been consistent with government decisions to limit its hours of operation and use multiple warning signs. These factors limit the ability of the mobile speed camera program to effectively deliver a broad general network deterrence from speeding.

Many locations are needed to enable network-wide coverage and ensure MSC sessions are randomised and not predictable. However, there are insufficient locations available to operate MSCs that meet strict criteria for crash history, operator safety, signage and technical requirements. MSC performance would be improved if there were more locations.

A scheduling system is meant to randomise MSC location visits to ensure they are not predictable. However, a relatively small number of locations have been visited many times making their deployment more predictable in these places. The allocation of MSCs across the time of day, day of week and across regions is prioritised based on crash history but the frequency of location visits does not correspond with the crash risk for each location.

There is evidence of a reduction in fatal and serious crashes at the 30 best-performing MSC locations. However, there is limited evidence that the current MSC program in NSW has led to a behavioural change in drivers by creating a general network deterrence. While the overall reduction in serious injuries on roads has continued, fatalities have started to climb again. Compliance with speed limits has improved at the sites and locations that MSCs operate, but the results of overall network speed surveys vary, with recent improvements in some speed zones but not others.
There is no supporting justification for the number of hours of operation for the program. The rate of MSC enforcement (hours per capita) in NSW is less than Queensland and Victoria. The government decision to use multiple warning signs has made it harder to identify and maintain suitable MSC locations, and impeded their use for enforcement in both traffic directions and in school zones. 

The Department effectively consults with industry, training providers and government departments to identify skill needs, and targets subsidies to meet those needs. However, the Department does not have a robust, data driven process to remove subsidies from qualifications which are no longer a priority. There is a risk that some qualifications are being subsidised which do not reflect the skill needs of New South Wales. 
The Department needs to better use the data it has, and collect additional data, to support its analysis of priority skill needs in New South Wales, and direct funding accordingly.

In addition to subsidising priority qualifications, the Department promotes engagement in skills training by:

• funding scholarships and support for disadvantaged students
• funding training in regional and remote areas
• providing additional support to deliver some qualifications that the market is not providing.

The Department needs to evaluate these funding strategies to ensure they are achieving their goals. It should also explore why training providers are not delivering some priority qualifications through Smart and Skilled.

Training providers compete for funding allocations based on their capacity to deliver. The Department successfully manages the budget by capping funding allocated to each Smart and Skilled training provider. However, training providers have only one year of funding certainty at present. Training providers that are performing well are not rewarded with greater certainty.

The Department needs to improve its communication with prospective students to ensure they can make informed decisions in the VET market.

The Department also needs to communicate more transparently to training providers about its funding allocations and decisions about changes to the NSW Skills List. 

The Department relies on stakeholder proposals to update the NSW Skills List. Stakeholders include industry, training providers and government departments. These stakeholders, particularly industry, are likely to be aware of skill needs, and have a strong incentive to propose qualifications that address these needs. The Department’s process of collecting stakeholder proposals helps to ensure that it can identify qualifications needed to address material skill needs. 

It is also important that the Department ensures the NSW Skills List only includes priority qualifications that need to be subsidised by government. The Department does not have robust processes in place to remove qualifications from the NSW Skills List. As a result, there is a risk that the list may include lower priority skill areas. Since the NSW Skills List was first created, new additions to the list have outnumbered those removed by five to one.

The Department does not always validate information gathered from stakeholder proposals, even when it has data to do so. Further, its decision making about what to include on, or delete from, the NSW Skills List is not transparent because the rationale for decisions is not adequately documented. 

The Department is undertaking projects to better use data to support its decisions about what should be on the NSW Skills List. Some of these projects should deliver useful data soon, but some can only provide useful information when sufficient trend data is available. 

Recommendation

The Department should: 

• by June 2019, increase transparency of decisions about proposed changes to the NSW Skills List and improve record-keeping of deliberations regarding these changes
• by December 2019, use data more effectively and consistently to ensure that the NSW Skills List only includes high priority qualifications

Only qualifications on the NSW Skills List are eligible for subsidies under Smart and Skilled. As the Department does not have a robust process for removing low priority qualifications from the NSW Skills list, some low priority qualifications may be subsidised. 

The Department allocates the Smart and Skilled budget through contracts with Smart and Skilled training providers. Training providers that meet contractual obligations and perform well in terms of enrolments and completion rates are rewarded with renewed contracts and more funding for increased enrolments, but these decisions are not based on student outcomes. The Department reduces or removes funding from training providers that do not meet quality standards, breach contract conditions or that are unable to spend their allocated funding effectively. Contracts are for only one year, offering training providers little funding certainty. 

Smart and Skilled provides additional funding for scholarships and for training providers in locations where the cost of delivery is high or to those that cater to students with disabilities. The Department has not yet evaluated whether this additional funding is achieving its intended outcomes. 

Eight per cent of the qualifications that have been on the NSW Skills List since 2015 are not delivered under Smart and Skilled anywhere in New South Wales. A further 14 per cent of the qualifications that are offered by training providers have had no student commencements. The Department is yet to identify the reasons that these high priority qualifications are either not offered or not taken up by students.

Recommendation

The Department should:

• by June 2019, investigate why training providers do not offer, and prospective students do not enrol in, some Smart and Skilled subsidised qualifications 
• by December 2019, evaluate the effectiveness of Smart and Skilled funding which supplements standard subsidies for qualifications on the NSW Skills List, to determine whether it is achieving its objectives
• by December 2019, provide longer term funding certainty to high performing training providers, while retaining incentives for them to continue to perform well.

In a contestable market, it is important for consumers to have sufficient information to make informed decisions. The Department does not provide some key information to prospective VET students to support their decisions, such as measures of provider quality and examples of employment and further education outcomes of students completing particular courses. Existing information is spread across numerous channels and is not presented in a user friendly manner. This is a potential barrier to participation in VET for those less engaged with the system or less ICT literate.

The Department conveys relevant information about the program to training providers through its websites and its regional offices. However, it could better communicate some specific information directly to individual Smart and Skilled training providers, such as reasons their proposals to include new qualifications on the NSW Skills List are accepted or rejected. 

While the Department is implementing a communication strategy for VET in New South Wales, it does not have a specific communications strategy for Smart and Skilled which comprehensively identifies the needs of different stakeholders and how these can be addressed. 

Recommendation

By December 2019, the Department should develop and implement a specific communications strategy for Smart and Skilled to:

• support prospective student engagement and informed decision making
• meet the information needs of training providers 

The EPA also could not demonstrate that it has effective governance and oversight of its regulatory operations. The EPA operates in a complex regulatory environment where its regional offices have broad discretions for how they operate. The EPA has not balanced this devolved structure with an effective governance approach that includes appropriate internal controls to monitor the consistency or quality of its regulatory activities. It also does not have an effective performance framework that sets relevant performance expectations and outcome-based key performance indicators (KPIs) for its regional offices. 

These deficiencies mean that the EPA cannot be confident that it conducts compliance and enforcement activities consistently across the State and that licensees are complying with their licence conditions or the Act.

The EPA's reporting on environmental and regulatory outcomes is limited and most of the data it uses is self reported by industry. It has not set outcome-based key result areas to assess performance and trends over time. 

Elements of the EPA’s risk-based regulatory framework for water pollution and illegal solid waste disposal are consistent with the NSW Government Guidance for regulators to implement outcomes and risk-based regulation. There are important gaps in how the EPA implements its risk-based approach that limit the effectiveness of its regulatory response. The EPA could not demonstrate that it effectively regulates licensees because it has not established reliable practices that accurately and consistently detect licence non compliances or breaches of the Act and enforce regulatory actions.

The EPA lacks effective governance arrangements to support its devolved regional structure. The EPA's performance framework has limited and inconclusive reporting on regional performance to the EPA’s Chief Executive Officer or to the EPA Board. The EPA cannot assure that it is conducting its regulatory responsibilities effectively and efficiently. 

The EPA does not consistently evaluate its regulatory approach to ensure it is effective and efficient. For example, there are no set requirements for how EPA officers conduct mandatory site inspections, which means that there is a risk that officers are not detecting all breaches or non-compliances. The inconsistent approach also means that the EPA cannot rely on the data it collects from these site inspections to understand whether its regulatory response is effective and efficient. In addition, where the EPA identifies instances of non compliance or breaches, it does not apply all available regulatory actions to encourage compliance.

The EPA also does not have a systematic approach to validate self-reported information in licensees’ annual returns, despite the data being used to assess administrative fees payable to the EPA and its regulatory response to non-compliances. 

The EPA does not use performance frameworks to monitor the consistency or quality of work conducted across the State. The EPA has also failed to provide effective guidance for its staff. Many of its policies and procedures are out-dated, inconsistent, hard to access, or not mandated.

By 31 December 2018, to improve governance and oversight, the EPA should:
1.	implement a more effective performance framework with regular reports to the Chief Executive Officer and to the EPA Board on outcomes-based key result areas that assess its environmental and regulatory performance and trends over time
By 30 June 2019, to improve consistency in its practices, the EPA should:
2.	progressively update and make accessible its policies and procedures for regulatory operations, and mandate procedures where necessary to ensure consistent application
3.	implement internal controls to monitor the consistency and quality of its regulatory operations.

The requirements for setting licence conditions for water pollution are complex and require technical and scientific expertise. In August 2016, the EPA approved guidance developed by its technical experts in the Water Technical Advisory Unit to assist its regional staff. However, the EPA did not mandate the use of the guidance until mid-April 2018. Up until then, the EPA had left discretion to regional offices to decide what guidance their staff use. This meant that practices have differed across the organisation. The EPA is yet to conduct training for staff to ensure they consistently apply the 2016 guidance.

The EPA has not implemented any appropriate internal controls or quality assurance process to monitor the consistency or quality of licence conditions set by its officers across the State. This is not consistent with good regulatory practice.

The triennial 2016 audit of the Sydney drinking water catchment report highlighted that Lake Burragorang has experienced worsening water quality over the past 20 years from increased salinity levels. The salinity levels were nearly twice as high as in other storages in the Sydney drinking water catchment. The report recommended that the source and implication of the increased salinity levels be investigated. The report did not propose which public authority should carry out such an investigation. 

To date, no NSW Government agency has addressed the report's recommendation. There are three public authorities, the EPA, DPE and WaterNSW that are responsible for regulating activities that impact on water quality in the Sydney drinking water catchment, which includes Lake Burragorang. 

By 30 June 2019, to address worsening water quality in Lake Burragorang, the EPA should:
4.	(a) review the impact of its licensed activities on water quality in Lake Burragorang, and
	(b) develop strategies relating to its licensed activities (in consultation with other relevant NSW Government agencies) to improve and maintain the lake's water quality.

The EPA tailors its compliance monitoring approach based on the performance of licensees. This means that licensees that perform better have a lower administrative fee and fewer mandatory site inspections. 

However, this approach relies on information that is not complete or accurate. Sources of information include licensees’ annual returns, EPA site inspections and compliance audits, and pollution reports from the public. 

Licensees report annually to the EPA on their performance, including compliance against their licence conditions. The Act contains significant financial penalties if licensees provide false and misleading information in their annual returns. However, the EPA does not systematically or consistently validate information self-reported by licensees, or consistently apply regulatory actions if it discovers non-compliance. 

Self-reported compliance data is used in part to assess a licensed premises’ overall environmental risk level, which underpins the calculation of the administrative fee, the EPA’s site inspection frequency, and the licensee’s exposure to regulatory actions. It is also used to assess the load-based licence fee that the licensee pays.

The EPA has set minimum mandatory site inspection frequencies for licensed premises based on its assessed overall risk level. This is a key tool to detect non-compliance or breaches of the Act. However, the EPA has not issued a policy or procedures that define what these mandatory inspections should cover and how they are to be conducted. We found variations in how the EPA officers in the offices we visited conducted these inspections. The inconsistent approach means that the EPA does not have complete and accurate information of licensees’ compliance. The inconsistent approach also means that the EPA is not effectively identifying all non-compliances for it to consider applying appropriate regulatory actions.

The EPA also receives reports of pollution incidents from the public that may indicate non-compliance. However, the EPA has not set expected time frames within which it expects its officers to investigate pollution incidents. The EPA regional offices decide what to investigate and timeframes. The EPA does not measure regional performance regarding timeframes. 

The few compliance audits the EPA conducts annually are effective in identifying licence non-compliances and breaches of the Act. However, the EPA does not have a policy or required procedures for its regulatory officers to consistently apply appropriate regulatory actions in response to compliance audit findings. 

The EPA has not implemented any effective internal controls or quality assurance process to check the consistency or quality of how its regulatory officers monitor compliance across the State. This is not consistent with good regulatory practice.

To improve compliance monitoring, the EPA should implement procedures to:
5.	by 30 June 2019, validate self-reported information, eliminate hardcopy submissions and require licensees to report on their breaches of the Act and associated regulations in their annual returns
6.	by 31 December 2018, conduct mandatory site inspections under the risk-based licensing scheme to assess compliance with all regulatory requirements and licence conditions.

The EPA’s compliance policy and prosecution guidelines have a large number of available regulatory actions and factors which should be taken into account when selecting an appropriate regulatory response. The extensive legislation determining the EPA’s regulatory activities, and the devolved regional structure the EPA has adopted in delivering its compliance and regulatory functions, increases the risk of inconsistent compliance decisions and regulatory responses. A good regulatory framework needs a consistent approach to enforcement to incentivise compliance. 

The EPA has not balanced this devolved regional structure with appropriate governance arrangements to give it assurance that its regulatory officers apply a consistent approach to enforcement.

The EPA has not issued standard procedures to ensure consistent non-court enforcement action for breaches of the Act or non-compliance with licence conditions. Given our finding that the EPA does not effectively detect breaches and non-compliances, there is a risk that it is not applying appropriate regulatory actions for many breaches and non-compliances.

A recent EPA compliance audit identified significant non-compliances with incident management plan requirements. However, the EPA has not applied regulatory actions for making false statements on annual returns for those licensees that certified their plans complied with such requirements. The EPA also has not applied available regulatory actions for the non-compliances which led to the false or misleading statements.

By 31 December 2018 to improve enforcement, the EPA should:
7.	Implement procedures to systematically assess non-compliances with licence conditions and breaches of the Act and to implement appropriate and consistent regulatory actions.

The EPA did not achieve the NSW Illegal Dumping Strategy 2014–16 target of a 30 per cent reduction in instances of large scale illegal dumping in Sydney, the Illawarra, Hunter and Central Coast from 2011 levels. 

In the reporting period, the incidences of large scale illegal dumping more than doubled. The EPA advised that this increase may be the result of greater public awareness and reporting rather than increased illegal dumping activity. 

By June 2018, the EPA is due to implement one outstanding recommendation made by the ICAC but has not set a time for the other outstanding recommendation.  

The HealthRoster system is realising functional business benefits in the LHDs where it has been implemented. In these LHDs, financial control of payroll expenditure and rostering compliance with employment award conditions has improved. However, these LHDs are not measuring the value of broader benefits such as better management of staff leave and overtime.

NSW Health has addressed the lessons learned from earlier implementations to improve later implementations. Business benefits identified in the business case were well defined and are consistent with business needs identified by NSW Health. Three of four cluster 1 LHDs have been able to reduce the number of issues with rostering and payroll processes. LHDs in earlier implementations need to use HealthRoster more effectively to ensure they are getting all available benefits from it.

HealthRoster is taking six years longer, and costing $37.2 million more, to fully implement than originally planned. NSW Health attributes the increased cost and extended timeframe to the large scale and complexity of the full implementation of HealthRoster.

Business benefits identified for HealthRoster accurately reflect business needs.

NSW Health has a good understanding of the issues in previous rostering systems and has designed HealthRoster to adequately address these issues. Interviews with frontline staff indicate that HealthRoster facilitates rostering which complies with industrial awards. This is a key business benefit that supports the provision of quality patient care. We saw no evidence that any major business needs or issues with the previous rostering systems are not being addressed by HealthRoster.

In the period examined in this audit since 2015, NSW Health has applied appropriate project management and governance structures to ensure that risks and issues are well managed during HealthRoster implementation.

HealthRoster has had two changes to its budget and timeline. Overall, the capital cost for the project has increased from $88.6 million to $125.6 million (42 per cent) and has delayed expected project completion by four years from 2015 to 2019. NSW Health attributes the increased cost and extended time frame to the large scale and complexity of the full implementation of HealthRoster.

NSW Health has established appropriate governance arrangements to ensure that HealthRoster is successfully implemented and that it will achieve business benefits in the long term. During implementation, local steering committees monitor risks and resolve implementation issues. Risks or issues that cannot be resolved locally are escalated to the state-wide steering committee.

NSW Health has grouped local health districts, and other NSW Health organisations, into four clusters for implementation. This has enabled NSW Health to apply lessons learnt from each implementation to improve future implementations.

NSW Health has a benefits realisation framework, but it is not fully applied to HealthRoster.

NSW Health can demonstrate that HealthRoster has delivered some functional business benefits, including rosters that comply with a wide variety of employment awards.

NSW Health is not yet measuring and tracking the value of business benefits achieved. NSW Health did not have benefits realisation plans with baseline measures defined for LHDs in cluster 1 and 2 before implementation. Without baseline measures NSW Health is unable to quantify business benefits achieved. However, analysis of post-implementation reviews and interviews with frontline staff indicate that benefits are being achieved. As a result, NSW Health now includes defining baseline measures and setting targets as part of LHD implementation planning. It has created a benefits realisation toolkit to assist this process from cluster 3 implementations onwards.

NSW Health conducted post-implementation reviews for clusters 1 and 2 and found that LHDs in these clusters were not using HealthRoster to realise all the benefits that HealthRoster could deliver.

By September 2018, NSW Health should:

1. Ensure that Local Health Districts undertake benefits realisation planning according to the NSW Health benefits realisation framework
2. Regularly measure benefits realised, at state and local health district levels, from the statewide implementation of HealthRoster
3. Review the use of HealthRoster in Local Health Districts in clusters 1 and 2 and assist them to improve their HealthRoster related processes and practices.

By June 2019, NSW Health should:

4. Ensure that all Local Health Districts are effectively using demand based rostering.

While Infrastructure NSW and Transport for NSW managed the assessment processes effectively overall, they have not fully maintained all required documentation, such as conflict of interest registers. Keeping accurate records is important to support transparency and accountability to the public about funding allocation. The relevant agencies have taken steps to address this in the current funding rounds for both programs.

For both programs assessed, the relevant agencies have developed good strategies over time to support councils through the application process. These strategies include workshops, briefings and feedback for unsuccessful applicants. Transport for NSW and the Department of Premier and Cabinet have implemented effective tools to assist applicants in demonstrating the economic impact of their projects.

Infrastructure NSW is effective in identifying projects that are 'at‑risk' and assists in bringing them back on track. Infrastructure NSW has a risk‑based methodology to verify payment claims, which includes elements of good practice in grants administration. For example, it requires grant recipients to provide photos and engages Public Works Advisory to review progress claims and visit project sites.

Infrastructure NSW collects project completion reports for all Resources for Regions and Fixing Country Roads funded projects. Infrastructure NSW intends to assess benefits for both programs once each project in a funding round is completed. To date, no funding round has been completed. As a result, no benefits assessment has been done for any completed project funded in either program.

 

The project selection criteria are consistent with the program objectives set by the NSW Government, and the RIAP applied the criteria consistently. Probity and record keeping practices did not fully comply with the probity plans.

The assessment methodology designed by Infrastructure NSW is consistent with2 the program objectives and criteria. In the rounds that we reviewed, all funded projects met the assessment criteria.

Infrastructure NSW developed probity plans for both programs which provided guidance on the record keeping required to maintain an audit trail, including the use of conflict of interest registers. Infrastructure NSW and Transport for NSW did not fully comply with these requirements. The relevant agencies have taken steps to address this in the current funding rounds for both programs.

NSW Procurement Board Directions require agencies to ensure that they do not engage a probity advisor that is engaged elsewhere in the agency. Infrastructure NSW has not fully complied with this requirement. A conflict of interest arose when Infrastructure NSW engaged the same consultancy to act as its internal auditor and probity advisor.

While these infringements of probity arrangements are unlikely to have had a major impact on the assessment process, they weaken the transparency and accountability of the process.

Some councils have identified resourcing and capability issues which impact on their ability to participate in the application process. For both programs, the relevant agencies conducted briefings and webinars with applicants to provide advice on the objectives of the programs and how to improve the quality of their applications. Additionally, Transport for NSW and the Department of Premier and Cabinet have developed tools to assist councils to demonstrate the economic impact of their applications.

The relevant agencies provided feedback on unsuccessful applications to councils. Councils reported that the quality of this feedback has improved over time.

Recommendations

1. By June 2018, Infrastructure NSW should:
   • ensure probity reports address whether all elements of the probity plan have been effectively implemented.

2. By June 2018, Infrastructure NSW and Transport for NSW should:
   • maintain and store all documentation regarding assessment and probity matters according to the State Records Act 1998, the NSW Standard on Records Management and the relevant probity plans

Infrastructure NSW is responsible for overseeing and monitoring projects funded under Resources for Regions and Fixing Country Roads. Infrastructure NSW effectively manages projects to keep them on track, however it could do more to assure itself that all recipients have complied with funding deeds. Benefits and outcomes should also start to be measured and reported as soon as practicable after projects are completed to inform assessment of future projects.

Infrastructure NSW identifies projects experiencing unreasonable delays or higher than expected expenses as 'at‑risk'. After Infrastructure NSW identifies a project as 'at‑risk', it puts in place processes to resolve issues to bring them back on track. Infrastructure NSW, working with Public Works Advisory regional offices, employs a risk‑based approach to validate payment claims, however this process should be strengthened. Infrastructure NSW would get better assurance by also conducting annual audits of compliance with the funding deed for a random sample of projects.

Infrastructure NSW collects project completion reports for all Resources for Regions and Fixing Country Roads funded projects. It applies the Infrastructure Investor Assurance Framework to Resources for Regions and Fixing Country Roads at a program level. This means that each round of funding (under both programs) is treated as a distinct program for the purposes of benefits realisation. It plans to assess whether benefits have been realised once each project in a funding round is completed. As a result, no benefits realisation assessment has been done for any project funded under either Resources for Regions or Fixing Country Roads. Without project‑level benefits realisation, future decisions are not informed by the lessons from previous investments.

Recommendations

3. By December 2018, Infrastructure NSW should:
   • conduct annual audits of compliance with the funding deed for a random sample of projects funded under Resources for Regions and Fixing Country Roads
   • publish the circumstances under which unspent funds can be allocated to changes in project scope
   • measure benefits delivered by projects that were completed before December 2017
   • implement an annual process to measure benefits for projects completed after December 2017

4. By December 2018, Transport for NSW and Infrastructure NSW should:
   • incorporate a benefits realisation framework as part of the detailed application.

Conclusion

The Department of Education effectively and efficiently allocates grants to non‑government schools. Clarifying the objectives of grants, monitoring progress towards these objectives, and improving oversight, would strengthen accountability for the use of public funds by non‑government schools.

We tested a sample of grants provided to non‑government schools under all major schemes, and found that the Department of Education consistently allocates and distributes grants in line with its methodology. The Department has clear processes and procedures to efficiently collect data from schools, calculate the level of funding each school or System should receive, obtain appropriate approvals, and make payments.

We identified three areas where the Department could strengthen its management of grants to provide greater accountability for the use of public funds. First, the Department’s objectives for providing grants to non‑government schools are covered by legislation, intergovernmental agreements and grant guidelines. The Department could consolidate these objectives to allow for more consistent monitoring. Second, the Department relies on schools or System Authorities to engage a registered auditor to certify the accuracy of information on their enrolments and usage of grants. Greater scrutiny of the registration and independence of the auditors would increase confidence in the accuracy of this information. Third, the Department does not monitor how System Authorities reallocate grant funding to their member schools. Further oversight in this area would increase accountability for the use of public funds.

The Department effectively and efficiently allocates grants to non‑government schools. Strengthening its processes would provide greater assurance that the information it collects is accurate.

The Department provides clear guidelines to assist schools to provide the necessary census information to calculate per capita grants. Schools must get an independent external auditor, registered with ASIC, to certify their enrolment figures. The Department checks a sample of the auditors to ensure that they are registered with ASIC. Some other jurisdictions perform additional procedures to increase confidence in the accuracy of the census (for example, independently checking a sample of schools’ census data).

The Department accurately calculates and distributes per capita grants in accordance with its methodology. The previous methodology, used prior to 2018, was not updated frequently enough to reflect changes in schools' circumstances. Over 2014 to 2017, the Department provided additional grants to non‑government schools under the National Education Reform Agreement (NERA), to bring funding more closely in line with the Australian Department of Education and Training's Schooling Resource Standard (SRS). From 2018, the Department has changed the way it calculates per capita grants to more closely align with the Australian Department of Education and Training's approach.

The Department determines eligibility for grants by checking a school's registration status with NESA. However, NESA's approach to monitoring compliance with the registration requirements prioritises student learning and wellbeing requirements over the requirement for policies and procedures for proper governance. Given their importance to the appropriate use of government funding, NESA could increase its monitoring of policies and procedures for proper governance through its program of random inspections. Further, the Department and NESA should enter into a formal agreement to share information to more accurately determine the level of risk of non‑compliance at each school. This may help both agencies more effectively target their monitoring to higher‑risk schools.

By December 2018, the NSW Department of Education should:

2. Strengthen its processes to provide greater assurance that the enrolment and expenditure information it collects from non‑government schools is accurate. This should build on the work the Australian Government already does in this area.
3. Establish formal information‑sharing arrangements with the NSW Education Standards Authority to more effectively monitor schools' eligibility to receive funding.
    

By December 2018, the NSW Education Standards Authority should:

5. Extend its inspection practices to increase coverage of the registration requirement for policies and procedures for the proper governance of schools.
6. Establish formal information‑sharing arrangements with the NSW Department of Education to more effectively monitor schools' continued compliance with the registration requirements.

The Department’s current approach to managing grants to non‑government schools could be improved to provide greater confidence that funds are being spent in line with the objectives of the grant schemes.

The NSW Government provides funding to non‑government schools to improve student learning outcomes, and to support schooling choices by parents, but does not monitor whether these grants are achieving this. In addition, each grant program has specific objectives. The main objectives for the per capita grant program is to increase the rate of students completing Year 12 (or equivalent), and to improve education outcomes for students. While non‑government schools publicly report on some educational measures via the MySchool website, these measures do not address all the objectives. Strengthened monitoring and reporting of progress towards objectives, at a school level, would increase accountability for public funding. This may require the Department to formalise its access to student level information.

The Department has listed five broad categories of acceptable use for per capita grants, however, provides no further guidance on what expenditure would fit into these categories. Clarifying the appropriate use of grants would increase confidence that funding is being used as intended. Schools must engage an independent auditor, registered with ASIC, to certify that the funding has been spent. The Department could strengthen this approach by improving its processes to check the registration of the auditor, and to verify their independence.

The Department has limited oversight of funding provided to System Authorities (Systems). The Department provides grants to Systems for all their member schools. The Systems can distribute the grants to their schools according to their own methodology. Systems are not required to report to the Department how much of their grant was retained for administrative or centralised expenses. Increased oversight over how the Systems distribute this grant could provide increased transparency for the use of public funds by systems.

By December 2018, the NSW Department of Education should:

1. Establish and communicate funding conditions that require funded schools to:
   • adhere to conditions of funding, such as the acceptable use of grants, and accounting requirements to demonstrate compliance
   • report their progress towards the objectives of the scheme or wider Government initiatives
   • allow the Department to conduct investigations to verify enrolment and expenditure of funds
   • provide the Department with access to existing student level data to inform policy development and analysis.

4. Increase its oversight of System Authorities by requiring them to:
   • re‑allocate funds across their system on a needs basis, and report to the Department on this
   • provide a yearly submission with enough detail to demonstrate that each System school has spent their State funding in line with the Department's requirements.

All four agencies examined in the audit are taking steps to strengthen their risk culture. In these agencies, senior management communicates the importance of managing risk to their staff. They have risk management policies and funded central functions to oversee risk management. We also found many examples of risk management being integrated into daily activities.

That said, three of the four case study agencies could do more to understand their existing risk culture. As good practice, agencies should monitor their employees’ attitude to risk. Without a clear understanding of how employees identify and engage with risk, it is difficult to tell whether the 'tone' set by the executive and management is aligned with employee behaviours.

Our survey of risk culture found that three agencies could strengthen a culture of open communication, so that all employees feel comfortable speaking openly about risks. To support innovation, senior management could also do better at communicating to their staff the levels of risk they are willing to accept.

Some agencies are performing better than others in building their risk capabilities. Three case study agencies have reviewed the risk-related skills and knowledge of their workforce, but only one agency has addressed the gaps the review identified. In three agencies, staff also need more practical guidance on how to manage risks that are relevant to their day-to-day responsibilities.

NSW Treasury provides agencies with direction and guidance on risk management through policy and guidelines. Its principles-based approach to risk management is consistent with better practice. Nevertheless, there is scope for NSW Treasury to develop additional practical guidance and tools to support a better risk culture in the NSW public sector. NSW Treasury should encourage agency heads to form a view on the current risk culture in their agencies, identify desirable changes to that risk culture, and take steps to address those changes. 

In assessing an agency’s risk culture, we focused on four key areas:

Executive sponsorship (tone at the top)

In the four agencies we reviewed, senior management is communicating the importance of managing risk. They have endorsed risk management frameworks and funded central functions tasked with overseeing risk management within their agencies.

That said, we found that three case study agencies do not measure their existing risk culture. Without clear measures of how employees identify and engage with risk, it is difficult for agencies to tell whether employee's behaviours are aligned with the 'tone' set by the executive and management.

For example, in some agencies we examined we found a disconnect between risk tolerances espoused by senior management and how these concepts were understood by staff.

Employee perceptions of risk management

Our survey of staff indicated that while senior leaders have communicated the importance of managing risk, more could be done to strengthen a culture of open communication so that all employees feel comfortable speaking openly about risks. We found that senior management could better communicate to their staff the levels of risk they should be willing to accept.

Integration of risk management into daily activities and links to decision-making

We found examples of risk management being integrated into daily activities. On the other hand, we also identified areas where risk management deviated from good practice. For example, we found that corporate risk registers are not consistently used as a tool to support decision-making.

Support and guidance to help staff manage risks

Most case study agencies are monitoring risk-related skills and knowledge of their workforce, but only one agency has addressed the gaps it identified. While agencies are providing risk management training, surveyed staff in three case study agencies reported that risk management training is not adequate.

NSW Treasury provides agencies with direction and guidance on risk management through policy and guidelines. In line with better practice, NSW Treasury's principles-based policy acknowledges that individual agencies are in a better position to understand their own risks and design risk management frameworks that address those risks. Nevertheless, there is scope for NSW Treasury to refine its guidance material to support a better risk culture in the NSW public sector.

Recommendation

By May 2019, NSW Treasury should:

• Review the scope of its risk management guidance, and identify additional guidance, training or activities to improve risk culture across the NSW public sector. This should focus on encouraging agency heads to form a view on the current risk culture in their agencies, identify desirable changes to that risk culture, and take steps to address those changes.