Conclusion
1. The urban renewal projects on the former railway land are well targeted to support the objectives of the Program. However, there is insufficient evidence that the cost of the light rail will be justified by its contribution to Program objectives.

The planned uses of the former railway land achieve a balance between the economic and social objectives of the Program at a reasonable cost to the Government. HCCDC, and previously UrbanGrowth NSW, identified and considered options for land use that would best meet Program objectives. Required probity processes were followed for developments that involved financial transactions. Our audit did not assess the achievement of these objectives because none of the projects have been completed yet.

Analysis presented in the Program business case and other planning documents showed that the light rail would have small transport benefits and was expected to make a modest contribution to broader Program objectives. Analysis in the Program business case argued that despite this, the light rail was justified because it would attract investment and promote economic development around the route. The Program business case referred to several international examples to support this argument, but did not make a convincing case that these examples were comparable to the proposed light rail in Newcastle.

The audited agencies argue that the contribution of light rail cannot be assessed separately because it is a part of a broader Program. The cost of the light rail makes up around 53 per cent of the total Program funding. Given the cost of the light rail, agencies need to be able to demonstrate that this investment provides value for money by making a measurable contribution to the Program objectives.

2. Consultation and oversight were mostly effective during the implementation stages of the Program. There were weaknesses in both areas in the planning stages.

Consultations about the urban renewal activities from around 2015 onward followed good practice standards. These consultations were based on an internationally accepted framework and met their stated objectives. Community consultations on the decision to close the train line were held in 2006 and 2009. However, the final decision in 2012 was made without a specific community consultation. There was no community consultation on the decision to build a light rail.

The governance arrangements that were in place during the planning stages of the Program did not provide effective oversight. This meant there was not a single agreed set of Program objectives until 2016 and roles and responsibilities for the Program were not clear. Leadership and oversight improved during the implementation phase of the Program. Roles and responsibilities were clarified and a multi-agency steering committee was established to resolve issues that needed multi-agency coordination.

Analysis presented in Program business cases and other planning documents showed that the light rail would have small transport benefits and was expected to make a modest contribution to broader Program objectives. Analysis in the Program business case argued that despite this, the light rail was justified because it would attract investment and promote economic development around the route. The Program business case referred to several international examples to support this argument, but did not make a convincing case that these examples were comparable to the proposed light rail in Newcastle.

The business case analysis of the benefits and costs of light rail was prepared after the decision to build light rail had been made and announced. Our previous reports, and recent reports by others, have emphasised the importance of completing thorough analysis before announcing infrastructure projects. Some advice provided after the initial light rail decision was announced was overly optimistic. It included benefits that cannot reasonably be attributed to light rail and underestimated the scope and cost of the project.

The audited agencies argue that the contribution of light rail cannot be assessed separately because it is part of a broader Program. The cost of the light rail makes up around 53 per cent of the total Program funding. Given the high cost of the light rail, we believe agencies need to be able to demonstrate that this investment provides value for money by making a measurable contribution to the Program objectives.

Recommendations
For future infrastructure programs, NSW Government agencies should support economical decision-making on infrastructure projects by:

• providing balanced advice to decision makers on the benefits and risks of large infrastructure investments at all stages of the decision-making process
• providing scope and cost estimates that are as accurate and complete as possible when initial funding decisions are being made
• making business cases available to the public.​​​​​​

The planned uses of the former railway land achieve a balance between the economic and social objectives of the Program at a reasonable cost to the government.

The planned uses of the former railway land align with the objectives of encouraging people to visit and live in the city centre, creating attractive public spaces, and supporting growth in employment in the city. The transport benefits of the activities are less clear, because the light rail is the major transport project and this will not make significant improvements to transport in Newcastle.

The processes used for selling and leasing parts of the former railway land followed industry standards. Options for the former railway land were identified and assessed systematically. Competitive processes were used for most transactions and the required assessment and approval processes were followed. The sale of land to the University of Newcastle did not use a competitive process, but required processes for direct negotiations were followed.

Recommendation
By March 2019, the Hunter and Central Coast Development Corporation should:

• work with relevant stakeholders to explore options for increasing the focus on the heritage objective of the Program in projects on the former railway land. This could include projects that recognise the cultural and industrial heritage of Newcastle.

Consultations about the urban renewal activities followed good practice standards, but consultation on transport decisions for the Program did not.

Consultations focusing on urban renewal options for the Program included a range of stakeholders and provided opportunities for input into decisions about the use of the former railway land. These consultations received mostly positive feedback from participants. Changes and additions were made to the objectives of the Program and specific projects in response to feedback received. 

There had been several decades of debate about the potential closure of the train line, including community consultations in 2006 and 2009. However, the final decision to close the train line was made and announced in 2012 without a specific community consultation. HCCDC states that consultation with industry and business representatives constitutes community consultation because industry representatives are also members of the community. This does not meet good practice standards because it is not a representative sample of the community.

There was no community consultation on the decision to build a light rail. There were subsequent opportunities for members of the community to comment on the implementation options, but the decision to build it had already been made. A community and industry consultation was held on which route the light rail should use, but the results of this were not made public. 

Recommendation
For future infrastructure programs, NSW Government agencies should consult with a wide range of stakeholders before major decisions are made and announced, and report publicly on the results and outcomes of consultations. 

The governance arrangements that were in place during the planning stages of the Program did not provide effective oversight. Project leadership and oversight improved during the implementation phase of the Program.

Multi-agency coordination and oversight were ineffective during the planning stages of the Program. Examples include: multiple versions of Program objectives being in circulation; unclear reporting lines for project management groups; and poor role definition for the initial advisory board. Program ownership was clarified in mid-2016 with the appointment of a new Program Director with clear accountability for the delivery of the Program. This was supported by the creation of a multi-agency steering committee that was more effective than previous oversight bodies.

The limitations that existed in multi-agency coordination and oversight had some negative consequences in important aspects of project management for the Program. This included whole-of-government benefits management and the coordination of work to mitigate impacts of the Program on small businesses.

Recommendations
For future infrastructure programs, NSW Government agencies should: 

• develop and implement a benefits management approach from the beginning of a program to ensure responsibility for defining benefits and measuring their achievement is clear
• establish whole-of-government oversight early in the program to guide major decisions. This should include:
  • agreeing on objectives and ensuring all agencies understand these
  • clearly defining roles and responsibilities for all agencies
  • establishing whole-of-government coordination for the assessment and mitigation of the impact of major construction projects on businesses and the community.

By March 2019, the Hunter and Central Coast Development Corporation should update and implement the Program Benefits Realisation Plan. This should include:

• setting measurable targets for the desired benefits
• clearly allocating ownership for achieving the desired benefits
• monitoring progress toward achieving the desired benefits and reporting publicly on the results.

The Department effectively consults with industry, training providers and government departments to identify skill needs, and targets subsidies to meet those needs. However, the Department does not have a robust, data driven process to remove subsidies from qualifications which are no longer a priority. There is a risk that some qualifications are being subsidised which do not reflect the skill needs of New South Wales. 
The Department needs to better use the data it has, and collect additional data, to support its analysis of priority skill needs in New South Wales, and direct funding accordingly.

In addition to subsidising priority qualifications, the Department promotes engagement in skills training by:

• funding scholarships and support for disadvantaged students
• funding training in regional and remote areas
• providing additional support to deliver some qualifications that the market is not providing.

The Department needs to evaluate these funding strategies to ensure they are achieving their goals. It should also explore why training providers are not delivering some priority qualifications through Smart and Skilled.

Training providers compete for funding allocations based on their capacity to deliver. The Department successfully manages the budget by capping funding allocated to each Smart and Skilled training provider. However, training providers have only one year of funding certainty at present. Training providers that are performing well are not rewarded with greater certainty.

The Department needs to improve its communication with prospective students to ensure they can make informed decisions in the VET market.

The Department also needs to communicate more transparently to training providers about its funding allocations and decisions about changes to the NSW Skills List. 

The Department relies on stakeholder proposals to update the NSW Skills List. Stakeholders include industry, training providers and government departments. These stakeholders, particularly industry, are likely to be aware of skill needs, and have a strong incentive to propose qualifications that address these needs. The Department’s process of collecting stakeholder proposals helps to ensure that it can identify qualifications needed to address material skill needs. 

It is also important that the Department ensures the NSW Skills List only includes priority qualifications that need to be subsidised by government. The Department does not have robust processes in place to remove qualifications from the NSW Skills List. As a result, there is a risk that the list may include lower priority skill areas. Since the NSW Skills List was first created, new additions to the list have outnumbered those removed by five to one.

The Department does not always validate information gathered from stakeholder proposals, even when it has data to do so. Further, its decision making about what to include on, or delete from, the NSW Skills List is not transparent because the rationale for decisions is not adequately documented. 

The Department is undertaking projects to better use data to support its decisions about what should be on the NSW Skills List. Some of these projects should deliver useful data soon, but some can only provide useful information when sufficient trend data is available. 

Recommendation

The Department should: 

• by June 2019, increase transparency of decisions about proposed changes to the NSW Skills List and improve record-keeping of deliberations regarding these changes
• by December 2019, use data more effectively and consistently to ensure that the NSW Skills List only includes high priority qualifications

Only qualifications on the NSW Skills List are eligible for subsidies under Smart and Skilled. As the Department does not have a robust process for removing low priority qualifications from the NSW Skills list, some low priority qualifications may be subsidised. 

The Department allocates the Smart and Skilled budget through contracts with Smart and Skilled training providers. Training providers that meet contractual obligations and perform well in terms of enrolments and completion rates are rewarded with renewed contracts and more funding for increased enrolments, but these decisions are not based on student outcomes. The Department reduces or removes funding from training providers that do not meet quality standards, breach contract conditions or that are unable to spend their allocated funding effectively. Contracts are for only one year, offering training providers little funding certainty. 

Smart and Skilled provides additional funding for scholarships and for training providers in locations where the cost of delivery is high or to those that cater to students with disabilities. The Department has not yet evaluated whether this additional funding is achieving its intended outcomes. 

Eight per cent of the qualifications that have been on the NSW Skills List since 2015 are not delivered under Smart and Skilled anywhere in New South Wales. A further 14 per cent of the qualifications that are offered by training providers have had no student commencements. The Department is yet to identify the reasons that these high priority qualifications are either not offered or not taken up by students.

Recommendation

The Department should:

• by June 2019, investigate why training providers do not offer, and prospective students do not enrol in, some Smart and Skilled subsidised qualifications 
• by December 2019, evaluate the effectiveness of Smart and Skilled funding which supplements standard subsidies for qualifications on the NSW Skills List, to determine whether it is achieving its objectives
• by December 2019, provide longer term funding certainty to high performing training providers, while retaining incentives for them to continue to perform well.

In a contestable market, it is important for consumers to have sufficient information to make informed decisions. The Department does not provide some key information to prospective VET students to support their decisions, such as measures of provider quality and examples of employment and further education outcomes of students completing particular courses. Existing information is spread across numerous channels and is not presented in a user friendly manner. This is a potential barrier to participation in VET for those less engaged with the system or less ICT literate.

The Department conveys relevant information about the program to training providers through its websites and its regional offices. However, it could better communicate some specific information directly to individual Smart and Skilled training providers, such as reasons their proposals to include new qualifications on the NSW Skills List are accepted or rejected. 

While the Department is implementing a communication strategy for VET in New South Wales, it does not have a specific communications strategy for Smart and Skilled which comprehensively identifies the needs of different stakeholders and how these can be addressed. 

Recommendation

By December 2019, the Department should develop and implement a specific communications strategy for Smart and Skilled to:

• support prospective student engagement and informed decision making
• meet the information needs of training providers 

The strength of fraud control systems varies significantly across New South Wales local councils, and many councils we surveyed need to improve significantly. 

Most surveyed councils do not have fraud control plans that direct resources to mitigating the specific fraud risks they face. Few councils reported that they conduct regular risk assessments or health checks to ensure they respond effectively to the risks they identify. 

There are sector wide weaknesses that impact on the strength of councils' fraud control practice. Less than one-third of councils that responded to the survey:

• communicate their expectations about ethical conduct and responsibility for fraud control to staff 
• regularly train staff to identify and respond to suspected fraud
• inform staff or the wider community how to report suspected fraud and how reports made will be investigated.

The audit also identified a pattern of councils developing policies, procedures or systems without ensuring people understand them, or assessing that they work. This reduces the likelihood that staff will actually use them. 

In general, metropolitan and regional councils surveyed have stronger fraud control systems than rural councils. 

Newly amalgamated councils are operating with systems inherited from two or more pre-amalgamated councils. These councils are developing new systems for their changed circumstances.

Five councils surveyed reported that they did not comply with the Public Interest Disclosure Act 1994. 

Observations for the sector:
Councils should improve their fraud controls by:

• tailoring fraud control plans to their circumstances and specific risks
• systematically and regularly reviewing their fraud risks and fraud control systems to keep their plans up to-date
• effectively communicating fraud risks, and how staff and the community can report suspected fraud 
• ensuring that they comply with the Public Interest Disclosure Act 1994.

Recommendation:
That the Office of Local Government: 

• work with councils to ensure they comply with the Public Interest Disclosure Act 1994.
   

Despite several New South Wales state entities collecting data on suspected fraud, the cost, extent, and nature of fraud in local councils is not clear. 

There are weaknesses in data collection and categorisation. Several state entities receive complaints about councils. These entities often do not separate complaints about fraud from other complaint data, do not separate local council data from other public-sector data, and do not separate complaints about council decisions or councillors from complaints about council staff conduct. Complaints about one incidence of suspected fraud can also be reported multiple times. 

Collaboration between state entities and councils to address these weaknesses in data collection could provide a clearer picture to the public and councils on the incidence of suspected fraud. Better information may also help councils decide where to focus fraud control efforts and apply resources more effectively.

Including measures for fraud control strength and maturity in the OLG performance framework may also improve practice in councils. Further, OLG may want to consider how a revised Model Code could better drive fraud control practice in councils.

Recommendations
That the Office of Local Government:

•  work with state entities and councils to develop a common approach to how fraud complaints and incidences are defined and categorised so that they can:
  • better use data to provide a clearer picture of the level of fraud within councils
  • measure the effectiveness of, and drive improvement in councils' fraud controls systems

The HealthRoster system is realising functional business benefits in the LHDs where it has been implemented. In these LHDs, financial control of payroll expenditure and rostering compliance with employment award conditions has improved. However, these LHDs are not measuring the value of broader benefits such as better management of staff leave and overtime.

NSW Health has addressed the lessons learned from earlier implementations to improve later implementations. Business benefits identified in the business case were well defined and are consistent with business needs identified by NSW Health. Three of four cluster 1 LHDs have been able to reduce the number of issues with rostering and payroll processes. LHDs in earlier implementations need to use HealthRoster more effectively to ensure they are getting all available benefits from it.

HealthRoster is taking six years longer, and costing $37.2 million more, to fully implement than originally planned. NSW Health attributes the increased cost and extended timeframe to the large scale and complexity of the full implementation of HealthRoster.

Business benefits identified for HealthRoster accurately reflect business needs.

NSW Health has a good understanding of the issues in previous rostering systems and has designed HealthRoster to adequately address these issues. Interviews with frontline staff indicate that HealthRoster facilitates rostering which complies with industrial awards. This is a key business benefit that supports the provision of quality patient care. We saw no evidence that any major business needs or issues with the previous rostering systems are not being addressed by HealthRoster.

In the period examined in this audit since 2015, NSW Health has applied appropriate project management and governance structures to ensure that risks and issues are well managed during HealthRoster implementation.

HealthRoster has had two changes to its budget and timeline. Overall, the capital cost for the project has increased from $88.6 million to $125.6 million (42 per cent) and has delayed expected project completion by four years from 2015 to 2019. NSW Health attributes the increased cost and extended time frame to the large scale and complexity of the full implementation of HealthRoster.

NSW Health has established appropriate governance arrangements to ensure that HealthRoster is successfully implemented and that it will achieve business benefits in the long term. During implementation, local steering committees monitor risks and resolve implementation issues. Risks or issues that cannot be resolved locally are escalated to the state-wide steering committee.

NSW Health has grouped local health districts, and other NSW Health organisations, into four clusters for implementation. This has enabled NSW Health to apply lessons learnt from each implementation to improve future implementations.

NSW Health has a benefits realisation framework, but it is not fully applied to HealthRoster.

NSW Health can demonstrate that HealthRoster has delivered some functional business benefits, including rosters that comply with a wide variety of employment awards.

NSW Health is not yet measuring and tracking the value of business benefits achieved. NSW Health did not have benefits realisation plans with baseline measures defined for LHDs in cluster 1 and 2 before implementation. Without baseline measures NSW Health is unable to quantify business benefits achieved. However, analysis of post-implementation reviews and interviews with frontline staff indicate that benefits are being achieved. As a result, NSW Health now includes defining baseline measures and setting targets as part of LHD implementation planning. It has created a benefits realisation toolkit to assist this process from cluster 3 implementations onwards.

NSW Health conducted post-implementation reviews for clusters 1 and 2 and found that LHDs in these clusters were not using HealthRoster to realise all the benefits that HealthRoster could deliver.

By September 2018, NSW Health should:

1. Ensure that Local Health Districts undertake benefits realisation planning according to the NSW Health benefits realisation framework
2. Regularly measure benefits realised, at state and local health district levels, from the statewide implementation of HealthRoster
3. Review the use of HealthRoster in Local Health Districts in clusters 1 and 2 and assist them to improve their HealthRoster related processes and practices.

By June 2019, NSW Health should:

4. Ensure that all Local Health Districts are effectively using demand based rostering.

All four agencies examined in the audit are taking steps to strengthen their risk culture. In these agencies, senior management communicates the importance of managing risk to their staff. They have risk management policies and funded central functions to oversee risk management. We also found many examples of risk management being integrated into daily activities.

That said, three of the four case study agencies could do more to understand their existing risk culture. As good practice, agencies should monitor their employees’ attitude to risk. Without a clear understanding of how employees identify and engage with risk, it is difficult to tell whether the 'tone' set by the executive and management is aligned with employee behaviours.

Our survey of risk culture found that three agencies could strengthen a culture of open communication, so that all employees feel comfortable speaking openly about risks. To support innovation, senior management could also do better at communicating to their staff the levels of risk they are willing to accept.

Some agencies are performing better than others in building their risk capabilities. Three case study agencies have reviewed the risk-related skills and knowledge of their workforce, but only one agency has addressed the gaps the review identified. In three agencies, staff also need more practical guidance on how to manage risks that are relevant to their day-to-day responsibilities.

NSW Treasury provides agencies with direction and guidance on risk management through policy and guidelines. Its principles-based approach to risk management is consistent with better practice. Nevertheless, there is scope for NSW Treasury to develop additional practical guidance and tools to support a better risk culture in the NSW public sector. NSW Treasury should encourage agency heads to form a view on the current risk culture in their agencies, identify desirable changes to that risk culture, and take steps to address those changes. 

In assessing an agency’s risk culture, we focused on four key areas:

Executive sponsorship (tone at the top)

In the four agencies we reviewed, senior management is communicating the importance of managing risk. They have endorsed risk management frameworks and funded central functions tasked with overseeing risk management within their agencies.

That said, we found that three case study agencies do not measure their existing risk culture. Without clear measures of how employees identify and engage with risk, it is difficult for agencies to tell whether employee's behaviours are aligned with the 'tone' set by the executive and management.

For example, in some agencies we examined we found a disconnect between risk tolerances espoused by senior management and how these concepts were understood by staff.

Employee perceptions of risk management

Our survey of staff indicated that while senior leaders have communicated the importance of managing risk, more could be done to strengthen a culture of open communication so that all employees feel comfortable speaking openly about risks. We found that senior management could better communicate to their staff the levels of risk they should be willing to accept.

Integration of risk management into daily activities and links to decision-making

We found examples of risk management being integrated into daily activities. On the other hand, we also identified areas where risk management deviated from good practice. For example, we found that corporate risk registers are not consistently used as a tool to support decision-making.

Support and guidance to help staff manage risks

Most case study agencies are monitoring risk-related skills and knowledge of their workforce, but only one agency has addressed the gaps it identified. While agencies are providing risk management training, surveyed staff in three case study agencies reported that risk management training is not adequate.

NSW Treasury provides agencies with direction and guidance on risk management through policy and guidelines. In line with better practice, NSW Treasury's principles-based policy acknowledges that individual agencies are in a better position to understand their own risks and design risk management frameworks that address those risks. Nevertheless, there is scope for NSW Treasury to refine its guidance material to support a better risk culture in the NSW public sector.

Recommendation

By May 2019, NSW Treasury should:

• Review the scope of its risk management guidance, and identify additional guidance, training or activities to improve risk culture across the NSW public sector. This should focus on encouraging agency heads to form a view on the current risk culture in their agencies, identify desirable changes to that risk culture, and take steps to address those changes.