The Auditor-General for New South Wales, Margaret Crawford, released a report today examining the effectiveness of Service NSW’s handling of customers’ personal information to ensure its privacy.

The audit found that Service NSW is not effectively handling personal customer and business information to ensure its privacy. Service NSW continues to use business processes that pose a risk to the privacy of personal information. This includes the routine emailing of personal information between Service NSW service centres and other agencies, which is one of the processes that contributed to the data breach earlier this year. The audit found that previously identified risks and recommended solutions had not been implemented on a timely basis.

The Auditor-General made eight recommendations aimed at ensuring improved processes, technologies, and governance arrangements for how Service NSW handles customers’ personal information.

The Hon. Victor Dominello, MP, Minister for Customer Service, requested this audit under section 27(B)(3)(c) of the Public Finance and Audit Act 1983 following public reports in May 2020 of a cyber security attack which had led to a breach of Service NSW customer information. This audit also included the Department of Customer Service which supports Service NSW with privacy, risk and governance functions.

Service NSW was established in 2013 with the intention that it would, over time, 'become the primary interaction point for customers accessing New South Wales Government transaction services'.

Service NSW's functions are set out in the Service NSW (One stop Access to Government Services) Act 2013. This legislation allows for other NSW Government agencies to delegate to and enter into agreements with the Chief Executive Officer of Service NSW in order for Service NSW to undertake service functions for the agency.

Service NSW now has agreements with 36 NSW Government client agencies to facilitate over 1,200 types of interactions and transactions for the community.

The nature of each agreement between Service NSW and its client agencies varies. Some client agencies have delegated authority to allow Service NSW staff to conduct transactions on their behalf in the agencies' systems. Other arrangements do not include the same degree of delegation. In these cases, Service NSW provides services such as responding to enquiries and validating documents.

In addition, Service NSW conducts transactions for its own programs, such as the Seniors Card. Personal information for these programs, as well as information for customers' MyServiceNSW accounts, are stored by Service NSW on its Salesforce Customer Relationship Management (CRM) system.

In March 2020, Service NSW suffered two cyber security attacks in short succession. Technical analysis undertaken by the Department of Customer Service (DCS) concluded that these attacks resulted from a phishing exercise through which external threat actors gained access to the email accounts of 47 staff members. These attacks resulted in the breach of a large amount of personal customer information that was contained in these email accounts. See Section 1.1 for further details.

This audit is being conducted in response to a request from the Hon. Victor Dominello, Minister for Customer Service, under section 27B(3)(c) of the Public Finance and Audit Act 1983. Minister Dominello requested that the Auditor General conduct a performance audit in relation to Service NSW's handling of sensitive customer and business information.

This audit assessed how effectively Service NSW handles personal customer and business information to ensure its privacy.

It addressed the following:

• Does Service NSW have processes and governance in place to identify and manage risks to the privacy of personal customer and business information?
• Does Service NSW have policies, processes and systems in place that support the effective handling of personal customer and business information to ensure its privacy?
• Has Service NSW effectively implemented its policies, processes and systems for managing personal customer and business information?

1. Key findings

Service NSW identifies privacy risks, but the controls and processes it put in place to mitigate these privacy risks were not adequate to prevent or limit the extent of the data breach that occurred in March 2020

Service NSW’s approach to risk management is framed by its Risk Management Guideline, which defines 'privacy and compliance' as one of the key types of risk for the agency. Service NSW's enterprise risk register identifies four strategic privacy related risks. Service NSW has set out a zero level appetite for privacy risk in its risk appetite statement.

Service NSW has assessed the adequacy of its controls for privacy risks as needing improvement. To be fully effective, the Risk Management Guideline says that these controls should have a focus that is ‘largely preventative and address the root causes’.

One of the business processes that was a key contributing factor to the data breach was the emailing of personal information by Service NSW staff to client agencies.

This process had been identified as a risk prior to the breach and some steps had been put in place to mitigate the risk. In particular, staff were required to manually delete emails that contained personal information. However, these measures were ineffective in preventing the breach, as the external threat actors still gained access to 47 staff email accounts that contained a large amount of personal information.

It is unclear why Service NSW did not effectively mitigate this risk prior to the breaches. However, Service NSW has advised that it implemented measures in June and October 2020 to automatically archive emails likely to contain personal information. This is expected to limit the quantity of information retained in email accounts for extended periods.

Service NSW has not put in place any technical or other solutions to avoid Service NSW staff having to scan and email personal information to some client agencies. Urgent action is needed to remove the requirement for staff to email personal information to client agencies, thereby mitigating the risk inherent in sending and storing this information using email.

There are weaknesses in the general IT and security controls implemented by Service NSW over its Salesforce CRM system, which holds the personal information of over four million customers

There are weaknesses in the general IT and security controls implemented by Service NSW over its Salesforce CRM system. These weaknesses include deficiencies in governance of role based access, monitoring and audit of staff access, and partitioning of program specific transaction information. These deficiencies create an increased risk of unauthorised access to the personal information of over four million customers which is stored in this system.

In addition, there is an absence of important controls to safeguard customers' privacy, such as multi factor authentication and reviewable logs of access history to their information. Such controls, when properly implemented, would enhance the control that customers are able to exercise over their personal information.

A privacy impact assessment conducted on Service NSW’s Salesforce CRM system in 2015 recommended that the system include the ability for customers to review access history to their personal information, as well as the option for customers to apply multi factor authentication to their accounts. While both these recommendations appeared positively received by Service NSW, neither have been implemented.

Since its inception, Service NSW’s use of Salesforce has extended to storing transaction data, particularly for transactions for which Service NSW is responsible, such as the Seniors Card. It also holds details of over four million MyServiceNSW account holders, including name, email address and phone number, and optional address details. It was not originally intended for the system to hold this volume and nature of customer information.

Lines of responsibility for meeting privacy obligations are unclear between Service NSW and its client agencies

Service NSW's privacy management plan does not clearly set out the privacy obligations of Service NSW and its client agencies. It sets out that 'compliance with the privacy principles will primarily be the responsibility of that [client] agency'. However, Service NSW has its own obligations under the security principles of the Privacy and Personal Information Protection Act 1998 (PPIP Act) to take reasonable steps to prevent unauthorised access to personal information, which is not made clear in the privacy management plan.

The agreements between Service NSW and client agencies reviewed for this audit only include general and high level references to privacy. Most do not include details of each parties' privacy responsibilities such as: which agency will provide the customer with a privacy notice explaining how their personal information will be handled, how personal information will be kept secure, how long Service NSW will retain information, what processes will be followed for internal reviews, and what specific planning is in place to respond to data breaches.

Service NSW's privacy management plan has not been updated to include new programs and governance changes

Service NSW's privacy management plan includes most of the matters required by law or good practice, with some exceptions. It does not explain any exemptions that the agency commonly relies on under the PPIP Act and does not address any health information that Service NSW may handle. It had also not been updated to reflect governance changes and the fact that, at the time this audit commenced, Service NSW was disclosing the content of internal review applications (the formal expression for 'complaints') to the Department of Customer Service (DCS). These governance changes were part of the centralisation of Service NSW's corporate support functions into DCS in late 2019, though internal review staff were seconded back into Service NSW during the course of this audit.

The current July 2019 privacy management plan has also not been updated since the rollout of a number of major new initiatives in 2020. These include 2019–20 bushfire emergency recovery initiatives (such as small business grants) and COVID 19 pandemic response initiatives (such as small business grants, border permits and the COVID safe check in app).

Service NSW routinely conducts privacy impact assessments for new initiatives, though privacy risks remain in legacy systems and processes

Service NSW routinely conducts privacy impact assessments for major new initiatives and the assessments reviewed for this audit largely accorded with good practice guidance.

Service NSW does not routinely review existing processes and systems to ensure that they are effective in ensuring the privacy of customer personal information. Business processes that create the highest risk to privacy, such as emailing of personal information, are more common in these longstanding legacy systems.

Service NSW's significant and rapid growth has outpaced the establishment of a robust control environment which has exacerbated privacy risks

Since it was established in 2013, Service NSW has experienced significant growth in the number and diversity of the types of transactions it provides, as well as the number of client agencies with which it works. The pace and extent of this growth has contributed to important controls not being properly implemented on a timely basis, which has heightened privacy risks, particularly in regard to existing, legacy systems and processes.

The pace of change and increasing demand for new program implementation has limited the opportunity for Service NSW, in collaboration with its client agencies, to revisit and redesign legacy business practices which pose a greater privacy risk. This includes the scanning and emailing of personal information.

While 2019–20 has seen additional demands placed on Service NSW in responding to the 2019–20 bushfire emergency and COVID 19 pandemic, it is the nature of the agency’s work that it operates in a fast paced and complex environment, where it is required to respond to multiple client agencies and stakeholders. Ensuring customer privacy should be integral to Service NSW’s business as usual operations.

2. Recommendations

Service NSW commissioned a number of external reviews and investigations stemming from the data breaches. The Auditor General's recommendations below have taken these other reviews into account. In order to offer assurance that it is appropriately protecting the privacy of its customers, Service NSW should address the full breadth of findings and recommendations made across all relevant reviews.

As a matter of urgency, Service NSW should:

1. in consultation with relevant client agencies and the Department of Customer Service, implement a solution for a secure method of transferring personal information between Service NSW and client agencies

2. review the need to store scanned copies of personal information and, if still required, implement a more secure method of storing this information and regular deletion of material.

By March 2021, Service NSW should:

3. ensure that all new agreements entered into with client agencies from 1 April 2021 address the deficiencies identified in this audit, including that they provide clarity on:

• the content and provision of privacy collection notices
• the terms by which personal information will be retained, stored, archived, and disposed of when no longer required
• steps that will be taken by each agency to ensure that personal information is kept secure
• the circumstances in which, and processes by which, applications for internal review will be referred by one agency to the other
• how identified breaches of privacy will be handled between agencies

4. in collaboration with the Department of Customer Service, review its privacy management plan to address the deficiencies raised in this audit, including:

• to clarify Service NSW's understanding of how responsibility for meeting privacy obligations are delineated between Service NSW and client agencies
• to better reflect the full scope and complexity of personal information handled by Service NSW
• to better explain how applications for internal review are handled between Service NSW and the Department of Customer Service
• to ensure regular ongoing review, either according to a schedule or when Service NSW experiences substantial change to its programs and handling of personal information

5. in consultation with the Department of Customer Service, review its policies and processes for the management of privacy risks, including to:

• ensure that there are appropriate mechanisms to escalate identified privacy risks from business units to the Executive Leadership Team
• ensure that there are action plans to address strategic privacy risks that are assessed as having ineffective controls.

By June 2021, Service NSW should:

6. address deficiencies in the controls over, and security for, its Salesforce customer relationship management and related systems that hold customer personal information, including:

• establish policies and processes for regular access reviews and monitoring of user activity in these systems, including for privileged users
• enable partitioning and role based access restrictions to personal information collected for different programs
• provide customers the choice to use multi factor authentication to further secure their MyServiceNSW accounts
• enable customers to view the transaction history of their personal information to detect possible mishandling.

By December 2021, Service NSW should:

7. ensure that all existing agreements with client agencies address the deficiencies identified in this audit, including that they provide clarity on:

• the content and provision of privacy collection notices
• the terms by which personal information will be retained, stored, archived, and disposed of when no longer required
• steps that will be taken by each agency to ensure that personal information is kept secure
• the circumstances in which, and processes by which, applications for internal review will be referred by one agency to the other
• how identified breaches of privacy will be handled between agencies

8. carry out a risk assessment of all processes, systems and transactions that involve the handling of personal information and undertake a privacy impact assessment for those that:

• are identified as high risk and have not previously had a privacy impact assessment
• have had major changes or updates since the privacy impact assessment was completed.

Appendix one – Responses from agencies

Appendix two – About the audit

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

The Auditor-General for New South Wales, Margaret Crawford, released a report today titled Education 2020. This report focuses on key observations and findings from the most recent audits of agencies in the Education cluster.

Unqualified audit opinions were issued for all cluster agencies’ financial statements. However, internal control deficiencies were identified across the cluster agencies, including deficiencies in the management of purchasing cards and 15 internal control issues that were repeated from the previous year.

The 2019–20 natural disasters caused widespread damage in both Northern and Southern NSW. The COVID‑19 pandemic further challenged agencies, requiring social distancing and other infection control measures which disrupted the traditional means of teaching students. Agencies have adjusted their operations to respond to these emergency events.

The TAFE Commission’s revenues 2019–20 were impacted by the pandemic. Lower enrolments and an increase in fee-free short courses offered during the year contributed to the result.

Read the PDF report

This report analyses the results of our audits of financial statements of entities within the Education cluster for the year ended 30 June 2020. The table below summarises our key observations and recommendations.

1. Financial reporting 

2. Audit observations

This report provides parliament and other users of the Education cluster’s financial statements with the results of our audits, our observations, analysis, conclusions and recommendations in the following areas:

• financial reporting
• audit observations
• the impact of emergencies and the COVID-19 pandemic.

Financial reporting is an important element of good governance. Confidence and transparency in public sector decision making are enhanced when financial reporting is accurate and timely.

The COVID-19 Legislation Amendment (Emergency Measures–Treasurer) Act 2020 amended legislation administered by the Treasurer to implement further emergency measures as a result of the COVID-19 pandemic. These amendments:

• allowed the Treasurer to authorise payments from the consolidated fund until the enactment of the 2020–21 budget – supporting the going concern assessments of cluster agencies
• revised budgetary, financial and annual reporting time frames – impacting the timeliness of financial reporting
• exempted certain statutory bodies and departments from preparing financial statements.

This chapter outlines our audit observations related to the financial reporting of agencies in the Education cluster for 2020, including any financial implications from the recent emergency events.

Appropriate financial controls help ensure the efficient and effective use of resources and administration of agency policies. They are essential for quality and timely decision making.

This chapter outlines our:

• observations and insights from our financial statement audits of agencies in the Education cluster. It also comments on our review of elements of the financial control framework applied by schools in NSW whose financial results form part of the Department of Education's (the Department) financial statements.
• assessment of how well cluster agencies adapted their systems, policies and procedures, and governance arrangements in response to recent emergencies.

Appendix one – List of 2020 recommendations

Appendix two – Status of 2019 and 2018 recommendations

Appendix three – Financial data

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

The Auditor-General for New South Wales, Margaret Crawford today released her report on the findings and recommendations from the 2019–20 financial audits that relate to internal controls and governance at 40 of the largest agencies in the NSW public sector.

The bushfire and flood emergencies and the COVID‑19 pandemic continue to have a significant impact on the people and public sector of New South Wales. The scale of the government response to these events has been significant. The report focuses on the effectiveness of internal controls and governance processes, including relevant agencies’ response to the emergencies. In particular, the report focuses on:

• financial and information technology controls
• business continuity and disaster recovery planning arrangements
• procurement, including emergency procurement
• delegations that support timely and effective decision-making.

Due to the ongoing impact of COVID‑19 agencies have not yet returned to a business‑as‑usual environment. ‘Agencies will need to assess their response to the recent emergencies and update their business continuity, disaster recovery and other business resilience frameworks to reflect the lessons learnt from these events’ the Auditor-General said.

The report noted that special procurement provisions were put in place to allow agencies to better respond to the COVID-19 pandemic. The Auditor-General recommended agencies update their procurement policies to reflect the current requirements of the NSW Procurement Framework and the emergency procurement requirements.

Read the PDF report

This report analyses the internal controls and governance of 40 of the largest agencies in the NSW public sector for the year ended 30 June 2020. These 40 agencies constitute an estimated 85 per cent of total expenditure for all NSW public sector agencies.

1. Internal control trends

2. Information technology controls

3. Business continuity and disaster recovery planning

4. Procurement, including emergency procurement

5. Delegations

6. Status of 2019 recommendations

Internal controls are processes, policies and procedures that help agencies to:

• operate effectively and efficiently
• produce reliable financial reports
• comply with laws and regulations
• support ethical government.

This chapter outlines the overall trends for agency controls and governance issues, including the number of audit findings, the degree of risk those deficiencies pose to the agency, and a summary of the most common deficiencies we found across agencies. The rest of this report presents this year’s controls and governance findings in more detail.

This chapter outlines our audit observations, conclusions and recommendations, arising from our review of agency controls to manage key financial systems.

This chapter outlines our audit observations, conclusions and recommendations, arising from our review of agency business continuity and disaster recovery planning arrangements.

This chapter outlines our audit observations, conclusions and recommendations, arising from our review of procurement agency procurement policies and procurement activity.

This chapter outlines our audit observations, conclusions and recommendations, arising from our review of agency compliance with financial and human resources delegations.

Appendix one – List of 2020 recommendations 

Appendix two – Status of 2019 recommendations

Appendix three – Cluster agencies

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

A report released today by the Auditor-General for New South Wales, Margaret Crawford found that select advertising campaigns conducted by Service NSW and the NSW Rural Fire Service met most requirements of the Government Advertising Act, regulations, Guidelines and other laws. However, the audit found that Service NSW inappropriately used its post campaign evaluation to measure sentiment towards and confidence in the NSW Government.  

While agency analysis shows that the ‘Cost of Living’ (phases 2 and 3)  and ‘How Fireproof is Your Plan?’ campaigns achieved most of their objectives, the campaign objectives and targets set by both agencies were not sufficient to measure all aspects of campaign effectiveness. 

The report makes two recommendations to the Department of Customer Service. The first is to review its guidance to ensure agencies are not using post campaign evaluations to measure sentiment towards the government. The second, to review its guidance and the new process of peer review to ensure they support agencies to comply with the Act, the regulations and the Guidelines. 

The Government Advertising Act 2011 requires the Auditor General to conduct an annual performance audit of one or more government agencies to see whether their advertising activities were carried out in an effective, economical and efficient manner and in compliance with the Government Advertising Act 2011.
 

Read full report (PDF)

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). This audit examined two campaigns run during the 2018–19 and 2019–20 financial years respectively:

• the 'Cost of Living' campaign run by Service NSW (phases 2 and 3 delivered in 2018–19)
• the 'How Fireproof Is Your Plan?' (Fireproof) campaign run by NSW Rural Fire Service (year two of a three-year campaign delivered in 2019–20).

Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.

In 2018–19, Service NSW delivered phases 2 and 3 of the 'Cost of Living' campaign. The Cost of Living advertising campaign aimed to build awareness of the help available to ease the cost of living for people under financial pressure including awareness of specific rebates that can be claimed. As part of the Cost of Living program, Service NSW developed a webpage designed as a single portal to access more than 40 NSW Government savings, rebates and initiatives (which originated from over 12 different agencies). It also launched the Cost of Living service which includes face to face meetings and phone interviews to help people claim rebates from the NSW Government. Phase 2 of the campaign ran from September 2018 to August 2019. Phase 3 of the campaign ran from January 2019 to July 2019. The budgets for phases 2 and 3 were $4.127 million and $934,800 respectively. See Appendix two for more details on this campaign.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign including radio transcripts, digital videos and display. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. See Appendix two for examples of campaign materials for this campaign.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

Post-campaign evaluations measured sentiment towards and confidence in the NSW Government

The post-campaign evaluation for phases 2 and 3 measured levels of confidence with the statement ‘the NSW Government has your best interests at heart’, despite the fact this was not a stated objective of the campaign. This is not an appropriate use of the post-campaign evaluation, which should measure the success of the campaign against its stated objectives. The post-campaign evaluation for phase 3 found that exposure to the campaign improved sentiment towards the government amongst those who did not have confidence in the NSW Government.

Service NSW advised that it was important to measure the sentiment of the advertising including the wording 'best interests' as it did not want the whole of government brand to be detrimental to customer engagement with applying for the rebates.

Following phase 2, Service NSW conducted analysis of media sentiment using the key words 'cost of living' and the names of the Premier, Treasurer and Minister for Customer Service. The analysis presented the level of positive, negative and neutral media sentiment. The Government Advertising Guidelines 2012 list the purposes that government advertising may serve which do not include improving the perception of the government. The inclusion of this analysis in Service NSW's post-campaign evaluation creates a risk that the results may be used for party political purposes.

Section 10 of the Act restricts agencies from carrying out a campaign after 26 January in the calendar year before the Legislative Assembly is due to expire and before the election for the Legislative Assembly in that year. Service NSW authorised a media agency to book media in line with the media plans for the campaign. The media plans for the campaign show that Service NSW did not authorise or plan to run any advertisements between 27 January 2019 and 23 March 2019.

Service NSW did not set targets for all rebates advertised in phase 2

Service NSW did not set targets for four of the seven rebates that were advertised as part of phase 2 of the campaign. These rebates were the Family Energy Rebate, Appliance Replacement Offer, National Parks Concession Offer and the Pensioner Travel Voucher. As a result, it was unable to evaluate whether the advertisements for these rebates were effective. Service NSW advised that at the time the campaign went to peer review, when campaign objectives are set, it did not know which rebates would be included in the advertisements.

Service NSW stated in its submission to the Department of Premier and Cabinet that it may change the creative content for phase 2 as it announced new initiatives and rebates. The peer review process should have ensured that Service NSW set targets for any additional rebates or savings it intended to advertise before that advertising commenced to ensure a strategic approach to the campaigns that clearly demonstrated anticipated benefits were in place.

The post-campaign evaluation for phase 2 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for some rebates advertised as part of phase 2 of the campaign. The objectives, targets and results for phase 2 are shown in Exhibit 5. In phase 2, Service NSW established baseline data on levels of awareness of government rebates during the peer review process. The baseline level of awareness for government rebates was 44 per cent. The level of awareness for specific rebates was 46 per cent for the Compulsory Third Party (CTP) green slip refund, and 21 per cent for both Active Kids and Toll Relief.

Post-campaign evaluation reports for phase 2 show that the campaign met its objective to raise awareness of NSW Government rebates, achieving a 16 per cent increase in awareness from 44 per cent to 51 per cent. The campaign did not meet its target to increase awareness of the CTP green slip refund by ten per cent.

Service NSW did not report the results of the uptake of the CTP green slip refund, Active Kids and Toll Relief in its post campaign effectiveness report submitted to the Department of Premier and Cabinet. However, other post-campaign evaluation documentation, which Service NSW advise was submitted to the Department of Premier and Cabinet, show that these targets were met.

Service NSW did not report to the Department of Premier and Cabinet on whether it achieved the target of a ten per cent increase of average monthly visits to the Cost of Living webpage. Service NSW reported that it had achieved an average of 11,753 visitors to the webpage per day during the campaign. These average daily results indicate that the target was met.

Exhibit 5: Phase 2 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. a) Increase awareness of rebates from the NSW Government by ten per cent.	Image
b) Increase average monthly visits to the Cost of Living webpage by ten per cent.	Image
2. Increase awareness of rebates and savings by ten per cent for:
CTP green slip refund	Image
Active Kids	Image
Toll Relief.	Image
3. Increase awareness that NSW Government initiatives relating to the cost of living are available via Service NSW by ten per cent.	Image
4. Increase the uptake of rebates and savings for the CTP green slip refund, Active Kids and Toll Relief by ten per cent.	Image

Key

Image

Yes

Image

Not Fully

The post-campaign evaluation for phase 3 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for the two rebates advertised as part of phase 3 of the campaign. The objectives, targets and results for phase 3 are shown in Exhibit 6.

In phase 3, Service NSW established baseline data on levels of awareness during the peer review process. The baseline level of awareness for government rebates was 44 per cent. This is the same baseline that was used to measure performance for phase 2 of the campaign. Service NSW did not set baselines for awareness and uptake of Energy Switch and Creative Kids as these were new services.

Post-campaign evaluation reports for phase 3 show that the campaign met its objective to raise awareness of NSW Government rebates by ten per cent, achieving a 30 per cent increase in awareness from 44 per cent to 57 per cent. The overall increase in message take-out was met with 43 per cent agreeing with the message that the NSW Government is taking steps to ease the cost of living. The campaign achieved awareness and uptake targets for the specific rebates included in phase 3, except for awareness of Creative Kids which achieved 28 per cent awareness, falling short of the 30 per cent awareness target.

Exhibit 6: Phase 3 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. Increase message takeout that ‘The NSW Government is taking steps to help ease the cost of living in NSW’ by ten per cent for those who can recall the campaign.
2. Increase awareness that the NSW Government has a range of rebates and savings by ten per cent.
3. Generate awareness with NSW residents aged 18+ of:
Energy Switch (15 per cent awareness)
Creative Kids (30 per cent awareness).
4. Create uptake of Energy Switch and Creative Kids (8,356 clicks on the Energy Switch website and 107,938 Creative Kids vouchers downloaded with 70 per cent conversion).

Key

Yes

Not Fully

The timing of campaign phases meant that it was difficult for Service NSW to evaluate each distinct campaign phase and reduced opportunities to incorporate learnings from previous phases

Service NSW commenced planning for phase 2 of the campaign while phase 1 was still underway. This limited the opportunity for Service NSW to incorporate learnings from phase 1 into phase 2. There was some overlap in the timing of phase 2 and the start of phase 3 of the campaign, making it difficult to evaluate the effectiveness of each distinct campaign phase. Both phases 2 and 3 had the same high-level outcome objective to raise awareness of rebates by ten per cent. The baseline measures that were used to evaluate performance for phase 3 were the same as those used to evaluate phase 2. As a result, Service NSW was not able to separately evaluate these two phases of the campaign. This is important given the budgets for phases 2 and 3 were $4.127million and $934,800 respectively.

Service NSW allocated 7.5 per cent of its media budget to communications with culturally and linguistically diverse (CALD) and Aboriginal audiences

The NSW Government CALD and Aboriginal Advertising Policy requires that agencies spend at least 7.5 per cent of an advertising campaign media budget on direct communications with CALD and Aboriginal audiences. Service NSW authorised a media company to book media in line with the media plans for the campaign. The media plans for phases 2 and 3 of the campaign indicate that Service NSW met this requirement, with 7.5 per cent of the budget allocated to these audiences in phase 2 and 10.4 per cent in phase 3.

The post campaign evaluation for phases 1 and 2 of the Cost of Living campaign contained a recommendation to look at other opportunities to reach CALD audiences. Effective communication with CALD audiences was particularly important in phase 3 of the campaign, where they made up 30 per cent of the target audience for the Creative Kids advertisement. The post-campaign analysis for phase 3 showed that the campaign performed well with some, but not all CALD audiences. The post-campaign analysis also showed low awareness and uptake with Aboriginal audiences. Pre-campaign focus groups in phase 3 found Aboriginal audiences had a negative reaction to the campaign tag line ‘NSW Government is helping with the cost of living’ however this tagline was still used in some advertisements in phase 3.

The cost-benefit analysis (CBA) for phase 2 did not accurately assess the benefits of the campaign and did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. The CBA conducted by Service NSW for phase 2 includes $8 million in benefits attributed to the advertisements for the Energy Switch tool and $6.9 million in benefits attributed to the advertisements for Creative Kids vouchers. These benefits should not have been included in the CBA for phase 2 as they were not included in this phase of the campaign. The CBA did not estimate the benefits of some other rebates and savings advertised in phase 2 of the campaign. This means that the CBA did not accurately assess the benefits of the campaign. Service NSW advised that at the time the CBA was developed it had not selected the rebates to be included in the campaign.

The Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits. The CBA developed as part of phase 2 identified using existing NSW Government communication channels as an alternative to advertising but did not assess the costs and benefits of this alternative.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Service NSW negotiated with a single creative agency in phase 2, making it difficult to demonstrate value for money

Agencies are required to obtain three quotes when procuring a creative agency on the prequalification scheme if the estimated cost of the creative content is greater than $150,000. In phase 2 of the campaign, Service NSW extended the contract with the creative agency used for phase 1 of the campaign and did not obtain three quotes despite the cost of the creative content for phase 2 being $731,480. The requirement to obtain three quotes was met in phase 1 when initially selecting this creative agency.

Service NSWs procurement policy details that direct negotiation may be appropriate where there is a compelling reason to renew or rollover a contract beyond temporal or convenience reasons or in the cases of a genuine emergency. In its briefing to the Chief Executive, Service NSW stated that this contract extension was sought due to the time-sensitive nature of the project and that if work was delayed by a tender process, Service NSW may not be able to meet marketing milestones and this could result in limited customer uptake. This reason is not a genuine emergency and is not compelling as it does not explain what consequences would occur if it did not meet the marketing milestones or if there was limited customer uptake.

Service NSW's procurement policy also states that under no circumstances must Service NSW employees enter into discussions with a supplier until the delegate has formally made their decision to enter into direct negotiation. Service NSW briefed the Chief Executive of Service NSW in relation to extending the contract on 5 September 2018. The briefing states that the creative agency had already begun developing creative content for phase 2 and Service NSW had already received quotes from the creative provider for the proposed work prior to 5 September 2018. Procurement sign-offs were not completed until 7 September 2018. The engagement of the creative provider prior to appropriate approvals was contrary to Service NSWs procurement policy.

The economy of the campaign may have been limited by not meeting the procurement requirements in phase 2. It is possible that the creative provider may have offered a more competitive rate if it was aware that Service NSW was seeking quotes from other creative providers. Additionally, it is possible that another creative provider could have provided better value for money.

In phase 3 of the campaign, the estimated cost of the creative exceeded $150,000 however Service NSW chose to contract two different creative agencies, and the cost for each agency fell below the threshold to obtain three quotes. Agencies are permitted to obtain one quote when using a creative provider on the prequalification scheme if the cost is between $50,000 to $150,000. Service NSW advised that it contracted two creative providers as two different project teams were responsible for the rebates, each with separate marketing budgets.

Service NSW allowed sufficient time for cost-efficient media placement

During the peer review process, the Department of Premier and Cabinet advised agencies about the time they should allow to ensure cost-efficient media placement. For example, the Department of Premier and Cabinet advised that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance.

Service NSW allowed sufficient time between the completion of the peer review process and the commencement of the first advertising. Service NSW signed the agreement with the approved Media Agency Services provider with sufficient time to achieve cost-efficient media placement for all types of media used in this campaign.

The campaign may have been misleading for some people who were not eligible for rebates

Advertisements we reviewed focused on the amount of savings that could be obtained from rebates, for example ‘Save up to $285’, and ended with a statement ‘To save, visit service.nsw.gov.au. This directed viewers to the Cost of Living website which contains eligibility information. However, the advertisements in phases 2 and 3 we reviewed did not contain any details on the eligibility for these rebates and not all advertisements stated that eligibility criteria apply. Service NSW advised that the eligibility criteria for each rebate is extensive and that it was not possible to include this in the creative material.

Post-campaign evaluations in phase 3 recommended that advertisements for Creative Kids should indicate eligibility (e.g. age criteria) as statements on savings have the potential to be misleading when not all viewers will be eligible for rebates. Social media analysis conducted following phase 2 showed ineligibility or inability to claim rebates or refunds caused anger for some respondents.

Some advertisements in phase 2 stated ‘we've got something for everyone’. However, as rebates were subject to eligibility criteria, it is possible that some residents in NSW would not be eligible for any rebates as part of the Cost of Living initiative. As such, this statement has the potential to be misleading.

The campaign included statements that underestimated the savings that some customers could obtain

The Guidelines require accuracy in the presentation of all facts, statistics, comparisons and other arguments. The Guidelines also require that all claims of fact included in government advertising campaigns must be able to be substantiated.

In phase 2, the possible savings customers could obtain for two rebates or savings exceeded the amounts stated in the advertising campaign. Exhibit 7 shows some advertisements in phase 2 which stated, ‘My Green Slip Saving Save up to $60’. However, the State Insurance Regulatory Authority website shows that savings for some types of motor vehicles under the 2017 CTP scheme exceed $60. The State Insurance Regulatory Authority website states that the average saving under this scheme has been $129. Service NSW advised that these advertisements were designed for regional markets and that it used different advertisements for metropolitan areas which contained different amounts of savings.

Some advertisements in phase 2 stated, ‘My Toll Relief save up to $700’. The Service NSW website states that drivers can obtain free vehicle registration if they have spent $1,352 or more in tolls in the previous financial year. The cost of registration for some vehicles exceeds $700. This means the savings detailed in the advertisement were lower than what some customers could actually save.

NSW Rural Fire Service conducted the 'How FireProof Is Your Plan?' (Fireproof) campaign. The Fireproof campaign is a three-year campaign which ran in 2018–19 (year one), 2019–20 (year two) and is planned for 2020–21 (year three). This audit examined year two of the campaign (2019–20).

The Fireproof campaign is a public safety campaign encouraging people to plan and prepare for bush fires across the summer period. The campaign aims to improve the quality of bush fire planning and preparation in the community and decrease the impact of fires on the community when they occur.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign for example radio advertisements, television commercials and digital displays. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. Examples of campaign materials are shown in Appendix two.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

NSW Rural Fire Service did not set targets for the second year of the campaign

The second year of the Fireproof campaign (2019–20) had the same objectives as the first year of the campaign (2018–19), however no specific targets were set for the second year. The advertising submission for the first year of the campaign (2018–19) details the targets for each objective as an increase of ten per cent against the baseline data to be achieved by March 2021, at the end of the three-year campaign.

The second year of the Fireproof campaign (2019–20) was one of the first campaigns approved under the new budget and peer review processes introduced by the Department of Customer Service in 2019–20. The new process for peer review introduced a new template for campaign submissions. The former template for campaign submissions contained more prompts for agencies to ensure the submission contained sufficient detail of campaign objectives, baseline measures, targets, dates for measurement and detail on how they would measure objectives. Despite this, the peer review process should have identified that NSW Rural Fire Service did not set targets for the second year of the campaign.

The 2016 Guidelines for Implementing NSW Government Evaluation Framework for Advertising and Communications requires campaign objectives to be SMART (specific, measurable, achievable, realistic and timed). NSW Rural Fire Service did not meet this requirement for year two of the Fireproof campaign.

Post-campaign evaluations showed increases against four out of five objectives, however there were no specific targets

NSW Rural Fire Service set three campaign objectives at the time it submitted the second year of the campaign (2019–20) to the Department of Customer Service for peer review. However, the post-campaign effectiveness report submitted to the Department of Customer Service measured campaign effectiveness against five campaign objectives. The objectives in the post-campaign effectiveness report were the same objectives set for the first year of the campaign, which is appropriate as this was a repeat campaign.

NSW Rural Fire Service achieved increases against four of their five objectives. However, as noted above there were no specific targets (such as percentage increases) against which performance of the 2019–20 campaign could be measured. Despite this, at the end of the second year, the Fireproof campaign had already achieved some of the targets that NSW Rural Fire Service had set for the end of the third year of the campaign. The post-campaign research showed that both audience recall and exposure to the campaign increased significantly from the prior year. The campaign objectives and results are shown in Exhibit 8.

For those people who already have a bush fire plan, the campaign aimed to increase the number of those plans which have included two or more elements from the Guide to Making a Bush Fire Survival Plan. Elements from the Guide to Making A Bush Fire Survival Plan include actions such as deciding what to take with you if you leave, ensuring you have the right equipment for defending your home and allocating responsibilities to members of a household. The post-campaign evaluation showed that the campaign did not achieve an increase against this objective for people who planned to stay and defend their property rather than leave.

Exhibit 8: Campaign objectives and results

Campaign objectives	Does the post-campaign evaluation show increases against the objective?
1. Continue to increase the number of people that have discussed and/or written a plan with regards to what they will do in the event of a fire.
2. Of those who indicate they have a plan, increase the number of people who have included two or more elements from the Guide to Making a Bush Fire Survival Plan:
for those who plan to leave
for those who plan to stay and defend.
3. Increase the frequency in completing preparation activities around a person’s property.
4. Increase the number of people who correctly assess it is their responsibility to complete preparation activities and enact their plan without direct intervention from emergency services.
5. Visits to MyFirePlan website.

Key

Yes

No

NSW Rural Fire Service achieved cost efficiencies by reusing creative content developed in the first year of the campaign

Total creative and production costs incurred in year one of the campaign were $1.08 million. Rather than commissioning new creative materials, NSW Rural Fire Service re-used the same creative content in year two of the campaign. NSW Rural Fire Service incurred $100,000 in creative and production costs in year two of the campaign and achieved cost-efficiencies by reusing the same creative developed in the prior year.

NSW Rural Fire Service allowed sufficient time for cost-efficient media placement and received free media placements

The Department of Customer Service advises agencies to work with media contacts to book media in advance to ensure a cost-efficient placement. Prior to 2019–20, the Department of Premier and Cabinet provided suggested timeframes for agencies to book media as part of the peer review process. For example, it advised agencies to book television six to 12 weeks in advance and book radio advertising two to eight weeks in advance. NSW Rural Fire Service allowed sufficient time for a cost-efficient media placement.

NSW Rural Fire Service received $4 million of free advertising time and space donated by media companies due to the extent and impact of the 2019–20 fire season.

The cost benefit analysis (CBA) did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. As part of the CBA, the Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits.

The CBA for the Fireproof campaign (year two) notes that the proposed campaign is one component of a broader community engagement strategy which has been developed over time and is based on research and evaluation. The CBA considers two options to achieve the objectives of the campaign. The first option is community engagement activities without an advertising campaign and the second option is community engagement activities alongside an advertising campaign. The CBA does not identify and assess the costs and benefits of both of the options in order to assess the most cost-efficient option.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Appendix one – Responses from agencies

Appendix two – About the campaigns

Appendix three – About the audit

Appendix four – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #342 - released 19 November 2020

This report examines whether NSW Health effectively planned and delivered major capital works to meet the demand for health services in New South Wales.

The report found that NSW Health has substantially expanded health infrastructure across New South Wales since 2015. However, the program was driven by Local Health District priorities without assessment of the State’s broader and future‑focussed health requirements.

The report found that unclear decision making roles and responsibilities between Health Infrastructure and the Ministry of Health limited the ability of NSW Health to effectively test and analyse investment options.

Project delays and budget overruns on some major projects indicate that Health Infrastructure's project governance, risk assessment and management systems could be improved.

The Auditor‑General recommends that NSW Health ensure its capital projects offer the greatest value to New South Wales by establishing effective policy guidance and enhancing project governance and management systems.

Read full report (PDF)

Since 2011–12, NSW Health has aimed to improve its facilities and build 'future focused' infrastructure. The NSW Government’s 2015–16 election commitments established a four-year $5.0 billion capital program for NSW Health to build and upgrade more than 60 hospitals and health services. The 2019–20 State Budget committed a further $10.1 billion over four years for another 29 projects. This is the largest investment to date on health capital works in New South Wales.

Recent reviews of infrastructure have recognised that population and demographic growth will require a change in the delivery and composition of health infrastructure, including considering greater use of non-traditional, non-capital health service options and assets.

To ensure that expenditure on capital works represents the best value for money, NSW Health's business cases need to be robust and supported by evidence that demonstrates they are worthy investments. The NSW Process of Facility Planning has been the main framework guiding the detailed planning and development of NSW Health's capital works proposals. This framework was developed by the then NSW Department of Health in 2010. Its aim is to ensure investment proposals are supported by rigorous planning processes that address health service needs and provide value for money.

Infrastructure projects of the complexity and scale being delivered by NSW Health carry inherent risks. For example, unplanned cost escalations can potentially impact on the State’s finances. Unforeseen delays can also reduce the intended benefits. The growth in the State’s health capital spend and project profile, means its exposure to such risks has increased over time.

The objective of this audit was to assess the effectiveness of planning and delivery of major capital works to meet demand for health services in New South Wales. To address this objective, the audit examined whether:

• the Ministry of Health has effective procedures for planning and prioritising investments in major health capital works
• Health Infrastructure develops robust business cases for initiated major capital works that reliably inform government decision making
• Health Infrastructure has effective project governance and management systems that support delivering projects on-time, within budget and achievement of intended benefits.

The audit focused on the Ministry of Health and Health Infrastructure – being the lead agencies within NSW Health responsible for prioritising, planning and delivering major health capital works across the State. The audit examined 13 business cases for eight discrete projects over a ten-year period.

Over the period of review, NSW Government policies for business case development and submission have emphasised that effective governance arrangements are critical to a proposal's successful implementation.

NSW Health's Process of Facility Planning similarly highlights the importance of effective governance and project management for achieving good outcomes. It prescribes a general governance structure managed by Health Infrastructure that can be tailored to the planning and delivery of health infrastructure projects greater than $10.0 million.

The three major hospital redevelopments examined in metropolitan, regional and rural areas had a combined Estimated Total Cost of more than $1.2 billion and comprised eight discrete projects and 13 separate business cases.

Almost all these projects experienced delivery challenges which impacted achievement of their original objectives and intended benefits. This is expected in complex and large-scale health infrastructure programs. However, in some projects the impacts were significant and resulted in substantial delays, unforeseen costs, and diversion of resources from other priority areas.

Our review of the selected case studies highlighted opportunities for enhancing governance and project management. Specifically, it indicates a need for improving transparency in the management of contingencies, risk management and assessments particularly relating to adverse site conditions and the selection of contractors. There is also a need to strengthen forward planning for options to address unfunded priorities within business cases that risk complicating the delivery of future project stages resulting in unforeseen costs and potentially avoidable budget overruns.

In February 2017, the Ministry's Capital Strategy Group approved the use of surplus funds of $13.76 million from Stage 1 of the Hornsby Ku-ring-gai Hospital Redevelopment for new works deemed needed to support Stage 2. Following this decision, Health Infrastructure finalised and submitted a business case addendum for Stage 1 to the Ministry in March 2017, addressing the new works comprising a two-storey building for medical imaging and paediatric floors. The business case addendum also addressed options to fit out and procure major medical imaging equipment. The Ministry approved the Stage 1 business case in July 2017, noting the Ministry's Capital Strategy Group had already approved the use of remaining Stage 1 funds to deliver the new works.

Stage 1 was completed in 2015, almost two years before the Stage 1 business case addendum was prepared in February 2017.

The Ministry's decision to approve the new works using $13.76 million of surplus Stage 1 funds did not comply with the NSW Treasury Circular TC 12/20. This policy establishes the Treasurer's approval must be sought and received before a new capital project with an Estimated Total Cost of $5.0 million or more can be approved by NSW Health. The Ministry therefore exceeded its delegated authority in making this decision, as it was not evident it had sought and received the Treasurer's approval prior to doing so.

Consequently, the surplus Stage 1 funds should not have been used by the Ministry to deliver new works in the circumstances. Instead, they should have been released from the Stage 1 project in accordance with established NSW Health procedures, and the Stage 1 Estimated Total Cost revised down accordingly. This did not occur, and NSW Health ultimately directed $11.0 million in surplus Stage 1 funds to the new works.

These circumstances indicate a need to strengthen transparency and accountability within NSW Health for the approval of new projects, and how contingency funds are used in the management of major health capital works. They also demonstrate the impact of weaknesses with options appraisal as the initial Stage 1 business case did not consider alternative options for addressing the initially unfunded works later covered by the Stage 1 business case addendum and ultimately funded from the Stage 1 contingency provision.

In addition to proposing the above-noted new works, the 2017 Stage 1 Business Case Addendum for the Hornsby-Ku-ring-gai development sought to retrospectively address the estimated funding gap of around $14.0 million for the internal fit out, supply of major medical imaging equipment, and cost to operate the medical imaging service at Hornsby Ku-ring-gai Hospital also not addressed in the originally Stage 1 business case.

The Stage 1 business case addendum considered various procurement options to purchase and run the medical imaging services ranging from State operation purchase options to private operation purchase options.

It recommended outsourcing the operation and provision of equipment to the private sector based on estimated savings to the public sector initially of around $650,000 per annum reducing over time to $270,000. The Ministry endorsed this option in June 2017, but it did not ultimately proceed.

A July 2018 report to the Executive Steering Committee on the project shows NSW Health later decided to deliver operation of the medical imaging unit 'traditionally' with an updated estimate of the cost at approximately $16.4 million. The report also shows the Ministry supported the costs now being met by the Northern Sydney Local Health District.

This means the funding gap previously identified in the Stage 1 business case addendum for fitting out the medical imaging building and supply of major medical equipment would need to be met fully by the State, representing a $16.4 million cost overrun for the project.

Examined reports to the Executive Steering Committee show this was largely funded by the Northern Sydney Local Health District via the disposal of land realising approximately $15.0 million in proceeds.

This initially unforeseen cost, along with the additional $11.0 million for the new works approved under the Stage 1 business case addendum, were ultimately merged with the Stage 2 project initially approved in 2017–18 with an Estimated Total Cost of $200 million.

The 2019–20 State Budget provided an additional $65.0 million for a further Stage 2A to deliver additional built capacity to support outpatient services, enhanced allied health services, re-housed community health services and the delivery of prioritised clinical services unfunded as part of Stage 2. The funds were approved based on an Investment Decision Template (IDT) that examined two options in addition to the base case representing scoping alternatives to the preferred master planned capital solution.

However, we found the IDT showed around 23 per cent of the $65.0 million sought (i.e. $15.0 million) was to be allocated to fund the deficit in Stage 2, which had arisen as a result of project delays due to adverse site conditions. This was not discussed in the IDT.

The February 2020 report to the Executive Steering Committee shows a combined Stage 2 and 2A final forecast cost of $292.6 million against a potential budget of $290.7 million representing an overall deficit for the project of around 0.6 per cent.

However, this favourable final budget position does not transparently show the funding challenges experienced over the project's implementation to-date. The three major budget issues include:

• inappropriate use of around $11.0 million in Stage 1 contingency for originally unfunded works contrary to Treasury policy
• the additional $16.4 million cost unforeseen in the Stage 1 business case for delivering medical imaging services mostly funded through the sale of land
• an additional $15.0 million from Stage 2A to cover the budget overrun in Stage 2 due to adverse site conditions.

The cumulative impact of these events is that Stages 1 and 2 of the Hornsby project cost approximately $42.4 million than it should have in the circumstances around 14 per cent more than what the revised combined Estimated Total Cost for both stages should have been after releasing the $11.0 million in surplus Stage 1 funds, with Stage 2 delayed by around 14 months.

Major construction projects often experience adverse site conditions which can be difficult to fully detect in advance. However, we found this was a common occurrence in the projects we examined sometimes with significant time and/or budget impacts indicating scope to enhance related risk and cost assessments. Specifically:

• Hornsby Ku-ring-gai Hospital Redevelopment Stage 2: adverse site conditions during demolition works resulted in an 11-month delay for delivering the medical imaging unit and 14-month delay completing Stage 2 main works including need for additional $15.0 million in funds to cover the resultant budget deficit for the project.
• Blacktown Mt Druitt Hospital Redevelopment Stage 2: adverse site conditions combined with project complexity delayed completion of the early works by approximately five months. This contributed to the delay in completing the main construction works which occurred around nine months later than planned in the business case.
• Dubbo Health Service Redevelopment Stages 3 and 4: Health Infrastructure advised adverse site conditions including asbestos containing materials and ground conditions delayed works for the main building with completion forecast for March 2021, around 21 months later than planned in the final business case. This resulted in the need for additional $13.5 million to cover increased construction costs and risks, increasing the Stage 3 and 4 forecast final cost from $150 million to $163.5 million as at February 2020.

These examples indicate a risk the cumulative impact of adverse site conditions may be substantial when measured across both time and Health Infrastructure's full delivery program. They also point to potential for Health Infrastructure to achieve efficiencies and improved outcomes from strengthening its approach to assessing and mitigating the risks from adverse site conditions.

Main construction works on Stage 1 of the Dubbo Health Service Redevelopment were completed in October 2015, approximately 13 months later than planned in the final business case. Delays were mainly due to insolvency of the early works contractor resulting in their departure from the project. The ensuing 11-month delay in completing the early works significantly impacted the overall schedule and delivery of main construction works.

The insolvency event was significant as it affected nine separate Health Infrastructure projects – three of which had yet to reach practical completion. It also affected state-funded projects in other sectors. It resulted in the need for additional funding of $11.5 million that was provided in the 2014–15 State Budget increasing the total Stage 1 and 2 budget from $79.8 million to $91.3 million.

Health Infrastructure’s analysis of lessons learned shows it worked actively to mitigate the impacts of the insolvency event across all affected projects. However, it also indicates a risk the lessons were mainly focused on mitigating the impacts after an insolvency event occurred rather than on prevention.

Although Health Infrastructure initially commissioned a financial assessment of the now insolvent early works contractor before engagement, it did not detect any risks of the impending insolvency and instead concluded the contractor was in a strong financial position. However, the contractor became insolvent shortly after commencement approximately seven months later. This indicates a risk of weaknesses in the assessment performed that was not explicitly addressed by the lessons learned.

Delivery of the main construction works were further impacted by disputes with the main works contractor over the scope of works for the renal unit resulting in Health Infrastructure terminating the contract in November 2016 following lengthy negotiations over several months.

The scope of works relating to the renal unit were ultimately transferred to Stages 3 and 4 and were delivered in December 2019, around five years later than originally planned in the business case.

Health Infrastructure advised the delay was ultimately beneficial to the project because the refurbishment works for the renal unit, initially scheduled for Stages 1 and 2, would have been demolished to accommodate the new Western Cancer Centre proposed after Stages 1 and 2 and currently being delivered in parallel with Stages 3 and 4.

Health Infrastructure advised the actual cost of Stages 1 and 2 was $84.7 million against the budget of $91.3 million. The residual $6.6 million relates to the renal works not delivered during Stage 1 and 2 and transferred to Stage 3 and 4.

Health Infrastructure advised the contractual provisions for mitigating insolvency events 'in-flight' are limited highlighting the importance of proactive and effective due diligence prior to engaging contractors for significant construction projects.

Health Infrastructure's 2017-20 Corporate Plan identifies the development of a quality framework to support delivery of future-focused outcomes as a key organisational priority. Related initiatives within the Corporate Plan describe a framework underpinned by a Quality Committee providing advice on:

• records management, to meet the requirements of the State Records Act 1998
• project assurance, to ensure future focused outcomes and enhance Health Infrastructure's Standards, Policies, Procedures and Guidelines, Templates and Design Guidance Notes
• knowledge management and library services, to promote and leverage from project learnings.

Although Health Infrastructure has some elements of a quality framework it is not yet fully in place. Health Infrastructure advised it had yet to establish the quality framework and related committee described in its Corporate Plan due in part to its focus on responding to the growth of its capital program.

Health Infrastructure's Development and Innovation team has been active in supporting continuous improvement in knowledge and project management including development of business cases. Although useful, these initiatives have relied heavily on leveraging and disseminating insights from Gateway reviews and have not formed part of a systematic quality and continuous improvement framework.

The limited focus on the quality of business cases is reflected in internal performance monitoring and reporting which focuses mainly on tracking the delivery of projects against internal benchmarks, often revised from the baselines in the business case, and expenditure against cashflow targets. There is no evident internal monitoring and/or reporting to the Chief Executive and Board on defined quality metrics linked to business case development and staff capability.

Performance reporting on balanced scorecard metrics has similarly focused mainly on process rather than quality and has been inconsistent in recent years.

Appendix one – Response from agency

Appendix two – About the audit

Appendix three – Performance auditing

Appendix four – Ministry of Health planning tools and guidelines

Appendix five – Streamlined investment decision process for Health Capital Projects

Appendix six – Timeline of business cases and relevant policy guidelines

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #338 - released 12 August 2020

This is a follow-up to the Auditor-General's November 2016 report on the CBD South East Sydney Light Rail project. This follow-up report assessed whether Transport for NSW has updated and consolidated information about project costs and benefits.

The audit found that Transport for NSW has not consistently and accurately updated project costs, limiting the transparency of reporting to the public.

The Auditor-General reports that the total cost of the project will exceed $3.1 billion, which is above the revised cost of $2.9 billion published in November 2019. $153.84 million of additional costs are due to omitted costs for early enabling works, the small business assistance package and financing costs attributable to project delays.

The report makes four recommendations to Transport for NSW to publicly report on the final project cost, the updated expected project benefits, the benefits achieved in the first year of operations and the average weekly journey times.

Read full report (PDF)

The CBD and South East Light Rail is a 12 km light rail network for Sydney. It extends from Circular Quay along George Street to Central Station, through Surry Hills to Moore Park, then to Kensington and Kingsford via Anzac Parade and Randwick via Alison Road and High Street.

Transport for NSW (TfNSW) is responsible for planning, procuring and delivering the Central Business District and South East Light Rail (CSELR) project. In December 2014, TfNSW entered into a public private partnership with ALTRAC Light Rail as the operating company (OpCo) responsible for delivering, operating and maintaining the CSELR. OpCo engaged Alstom and Acciona, who together form its Design and Construct Contractor (D&C).

On 14 December 2019, passenger services started on the line between Circular Quay and Randwick. Passenger services on the line between Circular Quay and Kingsford commenced on 3 April 2020.

In November 2016, the Auditor-General published a performance audit report on the CSELR project. The audit found that TfNSW would deliver the CSELR at a higher cost with lower benefits than in the approved business case, and recommended that TfNSW update and consolidate information about project costs and benefits and ensure the information is readily accessible to the public.

In November 2018, the Public Accounts Committee (PAC) examined TfNSW's actions taken in response to our 2016 performance audit report on the CSELR project. The PAC recommended that the Auditor-General consider undertaking a follow-up audit on the CSELR project. The purpose of this follow-up performance audit is to assess whether TfNSW has effectively updated and consolidated information about project costs and benefits for the CSELR project.

Appendix one – Response from agency

Appendix two – Governance and reporting arrangements for the CSELR

Appendix three – 2018 CSELR governance changes

Appendix four – About the audit

Appendix five – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #335 - released 11 June 2020

This report focuses on whether Destination NSW (DNSW) can demonstrate that its support for major events achieves value for money.

The audit found that DNSW’s processes for assessing and evaluating the major events it funds are mostly effective, but its public reporting does not provide enough transparency.

DNSW provides clear information to event organisers seeking funding and has a comprehensive methodology for conducting detailed event assessments. However, the reasons for decisions to progress events from the initial assessment to the detailed assessment stage are not documented in sufficient detail.

DNSW does not publish detailed information about the events it funds or the outcomes of these events. This means that members of the public are unable to see whether its activities achieve value for money. However, DNSW’s internal reporting to its key decision‑makers, including the CEO, the Board and the Minister is appropriate.

The Auditor-General made four recommendations to DNSW, aimed at improving the transparency of its activities, improving the documentation of decisions and certain compliance matters, and streamlining its approach to assessing and evaluating events that receive smaller amounts of funding.

Read full report (PDF)

Destination NSW (DNSW) provides funding to attract a range of major events to New South Wales, including high-profile professional sports matches and tournaments, musicals, art and museum exhibitions, and participation-focused events such as festivals and sports events that members of the public can enter. The NSW Government's rationale for providing funding is to encourage event organisers to hold events in New South Wales, and to ensure that events held in New South Wales maximise the potential for attracting overseas and interstate visitors.

This audit assessed whether DNSW can demonstrate that its support for major events achieves value for money. In making this assessment, the audit examined whether:

• DNSW effectively assesses proposals to support major events
• DNSW effectively evaluates the impact of its support for major events.

This audit focused on DNSW's work to attract major events to New South Wales. It did not assess DNSW's tourism promotion or development work, which includes developing tourism strategies, marketing and advertising campaigns, national and international partnerships, and regional programs.

Appendix one – Response from Destination NSW

Appendix two – About the audit 

Appendix three – Performance auditing

Copyright Notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #332 - released 9 April 2020.

This report outlines whether the Department of Customer Service (the department) has effective controls in place to ensure the integrity of data in the Births, Deaths and Marriages Register (the register), and to prevent unauthorised access and misuse.

The audit found that the department has processes in place to ensure that the information entered in the register is accurate and that any changes to it are validated. Although there are controls in place to prevent and detect unauthorised access to, and activity in the register, there were significant gaps in these controls. Addressing these gaps is necessary to ensure the integrity of information in the register.

The Auditor-General made nine recommendations to the department, aimed at strengthening controls to prevent and detect unauthorised access to, and activity in the register. These included increased monitoring of individuals who have access to the register and strengthening security controls around the databases that contain the information in the register.

The NSW Registry of Births Deaths and Marriages is responsible for maintaining registers of births, deaths and marriages in New South Wales as well as registering adoptions, changes of names, changes of sex and relationships. Maintaining the integrity of this information is important as it is used to confirm people’s identity and unauthorised access to it can lead to fraud or identity theft.

Read full report (PDF)

The NSW Registry of Births Deaths and Marriages (BD&M) is responsible for maintaining registers of births, deaths and marriages in New South Wales. BD&M is also responsible for registering adoptions, changes of name, changes of sex and relationships. These records are collectively referred to as 'the Register'. The Births, Deaths and Marriages Registration Act 1995 (the BD&M Act) makes the Registrar (the head of BD&M) responsible for maintaining the integrity of the Register and preventing fraud associated with the Register. Maintaining the integrity of the information held in the Register is important as it is used to confirm people's identity. Unauthorised access to, or misuse of the information in the Register can lead to fraud or identity theft. For these reasons it is important that there are sufficient controls in place to protect the information.

BD&M staff access, add to and amend the Register through the LifeLink application. While BD&M is part of the Department of Customer Service, the Department of Communities and Justice (DCJ) manages the databases that contain the Register and sit behind LifeLink and is responsible for the security of these databases.

This audit assessed whether BD&M has effective controls in place to ensure the integrity of data in the Births, Deaths and Marriages Register, and to prevent unauthorised access and misuse. It addressed the following:

• Are relevant process and IT controls in place and effective to ensure the integrity of data in the Register and the authenticity of records and documents?
• Are security controls in place and effective to prevent unauthorised access to, and modification of, data in the Register?

Appendix one – Response from agency

Appendix two – About the audit

Appendix three – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #330 - released 7 April 2020.

The State Insurance Regulatory Authority’s (SIRA) ‘green slip refund’ campaign, and the TAFE semester one 2018 student recruitment campaign, complied with most requirements of the Government Advertising Act 2011 and the Government Advertising Guidelines, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford.

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies has carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines).

This audit examined two campaigns conducted in 2017–18:

• the 'Green slip refund' campaign run by the State Insurance Regulatory Authority (SIRA)
• the semester one component of the 'TAFE NSW 2018 Student Recruitment Annual Campaign Program' run by the NSW TAFE Commission (TAFE).

Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.

The State Insurance Regulatory Authority (SIRA) conducted the 'Green slip refund' campaign between March and June 2018. SIRA ran this campaign to raise awareness of the Compulsory Third Party (CTP) refunds and reforms after the Motor Accidents Injuries Act 2017 commenced in December 2017. SIRA's view is that the reforms include a reduced cost for CTP insurance, benefits for at-fault drivers, reduced opportunity for fraud and attempts to lower insurance company profits. Green slip holders are also able to claim partial refunds on their 2017 green slip insurance premium. The campaign aimed to make green slip holders aware of the refunds available, encourage them to claim online and to inform people about the changes to the green slip scheme. The campaign focused on the first two of these objectives. The total cost of the campaign was $1.9 million. See Appendix two for more details on this campaign.

Campaign materials we reviewed did not breach section 6 of the Act

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit team found no breaches of section 6 of the Act in the campaign material we reviewed.
 
Before the start of the campaign, SIRA conducted a survey which asked people whether they agreed ‘that the NSW Government is helping to reduce the cost of living by making positive reforms to:

• reduce the cost of green slips
• reduce the cost of health insurance
• increase the number of jobs
• increase investment in the state.'

SIRA's initial submission to peer review listed one of the campaign objectives as improving the perception of the government as a positive reformer. DPC advised SIRA that this should not be included. SIRA removed this objective.

Even though SIRA appropriately removed this objective, the post-campaign evaluation still measured agreement with the above statements, three of which did not relate to this campaign or SIRA's responsibilities. SIRA advised that these three additional statements were included to provide a broader context for any change in the green slip campaign survey results. For example, if all four measures reported an increase in positive responses of roughly the same size, then the increase may have been due to factors other than the advertising campaign.

This is not an appropriate use of the post-campaign evaluation, which should measure the success of the campaign against its stated objectives. The Guidelines list the purposes that government advertising may serve and none of these relate to improving the perception of the government. The inclusion of the above questions in SIRA's post-campaign evaluation creates a risk that the results may be used for party political purposes.

The campaign met most targets, however some were not challenging to achieve

The post-campaign evaluation demonstrated that the campaign met the targets for 12 of its 13 objectives including the targets relating to raising awareness of the refunds and the proportion of people claiming their refunds online. A fourteenth objective, the percentage of people aware that they should contact SIRA after a road accident injury, did not have a target set, meaning that it is not possible to say whether the campaign had the desired impact in this case.

In August 2017, before the campaign commenced, SIRA conducted a survey to determine the baselines for some of its objectives. This is a good practice to support an effective post campaign evaluation process. The survey found that 20 per cent of people were aware of the green slip reforms. SIRA's objective was to raise this to 25 per cent, which represents a small gain relative to the proposed campaign expenditure. The campaign aimed for 40 per cent of motorists to be aware of refunds, which is very low given that this was the primary focus of the campaign. SIRA followed the advice of its survey provider when setting these targets. 

In the survey carried out after the campaign, 66 per cent of people were aware of the availability of green slip refunds for most motorists. The campaign also aimed to get 83 per cent of motorists to claim their refunds via online channels. It met this target, with a total of 84 per cent. Finally, 62 per cent of people in the post-campaign survey were aware of the green slip reforms. This result is discussed further below.

The overall target for total number of refunds claimed is 85 per cent of eligible drivers, that is to say CTP holders. SIRA will evaluate the results of this objective after the conclusion of the refund period in June 2019.

The campaign did little to inform the public about the broader green slip reforms

One objective of the green slip refund campaign was to inform the public about the green slip reforms. The final campaign creative material focused almost entirely on the green slip refunds rather than the range of other reforms. This was because the peer review raised concerns that the creative material was attempting to deliver too many messages. 
 
The campaign submission stated that the advertising campaign would raise awareness of the broader reforms to the CTP scheme, citing several examples such as reduced opportunities for fraud and reduced insurer profits. SIRA also advised the Minister for Finance, Services and Property that secondary messaging in the campaign would benefit public understanding of the reforms.

Some of the television and radio advertisements referred to ‘more protection’ or ‘better protection’ for people injured on New South Wales roads, however advertisements did not refer to other elements of the reforms. Other campaign creative materials contained messages solely relating to the green slip refund and made no further reference to the broader reforms. SIRA used other communication channels, such as giving wallet cards to health service providers, to spread these messages to people, particularly those who had been injured.

Sixty two per cent of people in the post-campaign survey were aware of the green slip reforms. SIRA asked these people which benefits they associated with the reforms. The results of this survey are in Exhibit 4. Seventy-one per cent of this sample identified the reduced costs of green slips as one of the changes, but awareness of other elements of the reforms remains low. Though 29 per cent of people perceive the reforms to make the green slip scheme ‘fairer’, no more than 15 per cent of people could list a specific benefit which did not relate to insurance prices.

Another campaign target was to ensure that people understood that they should contact SIRA in case of an injury. None of the campaign creative materials contained this information. SIRA did some limited work to inform the public about this through its social media channels. One of the pieces of creative material directed the reader to SIRA's website for further information on the reforms, which contained this information. During the campaign period, there was an increase in the number of calls received by SIRA's CTP Assist phone line. However, in the post-campaign evaluation, only two per cent of surveyed people identified that they should contact SIRA in case of an injury.

The media plan allowed sufficient time for cost-efficient media placement

During the peer review process, DPC provides advice to agencies about the time they should allow to ensure cost-efficient media placement. For example, DPC advise that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance.

SIRA allowed sufficient time between the completion of the peer review process and the commencement of the first advertising. SIRA signed the agreement with the approved Media Agency Services provider eight weeks before the campaign started, meaning that it could achieve cost-efficient media placement for all types of media used in this campaign.

SIRA directly negotiated with a single supplier, making it difficult to demonstrate value for money

SIRA directly negotiated with a single supplier to procure the campaign's creative material. A direct negotiation occurs when an agency negotiates with a proponent without first undergoing a competitive process. It is difficult to demonstrate value for money using direct negotiation due to the lack of competition. 

ICAC's 'Guidelines for managing risks in direct negotiations' (ICAC Guidelines) provide guidance on how to undertake direct negotiations. SIRA has a direct negotiation checklist that aligns to the ICAC Guidelines. The SIRA checklist advises that staff should confirm that existing New South Wales prequalification schemes cannot provide the procurement before undertaking a direct negotiation. SIRA did not do this.

To procure creative materials, agencies can access the Advertising and Digital Communication Services prequalification scheme (the prequalification scheme). Using the prequalification scheme allows agencies to quickly seek quotes from suppliers who have a demonstrated track record and expertise. While agencies are not required to use the prequalification scheme, the NSW Procurement Board advises that agencies should use prequalification schemes where they are available to promote competition. 

By using direct negotiation when the prequalification scheme was available, and by not seeking quotes from other suppliers, SIRA was acting in a way that reduced competition. This increases the risk that SIRA did not achieve value for money in its procurement of creative materials.

SIRA advised that it sought to ensure value for money by comparing the quote from its selected supplier with the amount spent on creative materials in other campaigns of similar size. SIRA did not document this analysis at the time or include it as part of the briefing note staff used to seek approval for undertaking direct negotiation. As a result, decision-makers were not fully informed when approving this engagement. 

SIRA reported in a briefing note that it engaged in direct negotiations because:

• it believed that the original timeframe did not allow for a competitive tender process
• the supplier had done previous work on a related campaign for SIRA
• the supplier provided sample work which received positive feedback from focus groups.

In July 2017, when peer review commenced, SIRA planned to launch the campaign in November 2017 to coincide with the beginning of the green slip reforms. SIRA believed that this timeframe was narrow enough to warrant entering direct negotiations. The ICAC Guidelines advise that a narrow timeframe is not a valid reason to enter into a direct negotiation. In late October 2017, the campaign launch was delayed until March 2018 to stagger the demand on the resources of Service NSW, which is administering the refund. 

The ICAC Guidelines also advise against re-appointing a supplier because it has performed previous work. Instead, agencies could consider previous experience as one of several factors when deciding between quotes. In cases where an agency asks a supplier to provide sample work, the ICAC Guidelines advise that agencies should request sample work from multiple potential suppliers to promote competition.

The campaign's cost benefit analysis complied with the Act and Guidelines 

The Act requires a cost benefit analysis (CBA) for any government advertising campaign likely to exceed $1.0 million in value. Section six of the Guidelines set out the requirements for a government advertising CBA. The campaign's CBA complied with the requirements of the Act and the Guidelines.

The campaign CBA could have demonstrated further cost effectiveness if it considered alternative media mixes as outlined in NSW Treasury's 'Cost Benefit Analysis Framework for Government Advertising and Information Campaigns'. This would also have been consistent with the Handbook.

The cluster Secretary signed the compliance certificate instead of the head of SIRA

The Act requires the head of the agency running the campaign to sign a compliance certificate. 

The Secretary of the Department of Finance, Services and Innovation, the cluster to which SIRA belongs, signed the campaign's compliance certificate. However, section 17(2) of the State Insurance and Care Governance Act 2015 states that SIRA is ‘for the purposes of any Act, a NSW Government agency.’ Given this, the Chief Executive of SIRA was responsible for signing the compliance certificate for this campaign.

This is a minor non-compliance with the Act because the Chief Executive had reviewed the campaign and recommended that the Secretary sign the compliance certificate.  

The NSW TAFE Commission (TAFE) ran the 'TAFE NSW 2018 Student Recruitment Annual Campaign Program' from November 2017 to September 2018. The aim of the campaign was to assist TAFE in achieving its 2018 student enrolment target by improving the perception of TAFE's brand and generating student enquiries. This is the first state-wide campaign run by TAFE operating under the One TAFE model. Previously, each TAFE Institute ran its own campaigns. The total budget of the campaign was $19.5 million. This audit examined only the semester one 2018 component of the campaign, which ran from November 2017 to April 2018 at a total cost of $9.5 million. See Appendix two for more details on this campaign.

The campaign materials we reviewed did not breach section 6 of the Act

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit team found no breaches of section 6 of the Act in the campaign material we reviewed.

The campaign achieved 16 of 24 objectives, but did not reach its enrolment target

The campaign had 24 objectives which had a target for semester one. TAFE set these targets using a combination of previous experience, corporate objectives and brand surveys.

The overall objective of the combined semester one and two campaigns was to support TAFE achieving its 2018 total enrolment target of 549,636. TAFE's semester one target was 361,350, which it did not achieve. This indicates that the campaign was not fully effective.

The campaign achieved 11 of its 16 output objectives. The output targets related to TAFE's media placements and ability to reach an audience efficiently. TAFE tracked progress against many of the campaign's output objectives daily. TAFE altered its media channels throughout the campaign meaning that some of the output objectives were not met because TAFE decided to focus on alternative media channels. The campaign also achieved all seven of its outcome objectives. The outcome objectives related to changing the public perception of TAFE.

TAFE's initial media plan allowed for efficient media placement

During the peer review process, DPC provides advice to agencies about the time they should allow to ensure cost-efficient media placement. For example, DPC advise that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance. 

While TAFE's initial media plan allowed sufficient time between the approval of the campaign and its launch, a delay in receiving final approval for the campaign meant TAFE could not purchase media placements until two months later than planned. Most purchases still remained within DPC's recommended timeframes, but Indigenous television advertisements and metropolitan out of home advertisements both fell outside DPC's recommended time periods by one week. These delays did not impact on TAFE's efficiency.

TAFE re-used many creative materials, achieving some cost-savings

Rather than commissioning new creative materials, TAFE re-used many creative materials from the previous campaign and supplemented these with a selection of new creative materials. TAFE advised that this led to a cost saving of approximately $130,000.
TAFE sought quotes from suppliers on the government's Advertising and Digital Communication Services prequalification scheme for two creative material contracts. These contracts covered updates to existing materials and a selection of new materials.

The campaign's cost-benefit analysis did not comply with three requirements of the Guidelines

The Act requires an agency to conduct a cost-benefit analysis (CBA) if the cost of an advertising campaign is likely to exceed $1.0 million. The Guidelines set out the requirements of this CBA. TAFE did not comply with three of these requirements, outlined in Exhibit 5.

In this circumstance, section 6.2 of the Guidelines required the CBA to identify the number of enrolments TAFE would expect if it did not advertise. TAFE advised us that it is not possible to say what this scenario would look like because there had always been some degree of advertising, however, this argument is not reflected in the CBA. 

TAFE used 2017 as the baseline in the CBA. In 2017, TAFE spent $13.2 million on advertising. As such, the CBA was only able to isolate the impact of the increased expenditure rather than the impact of the campaign's entire $19.5 million expenditure. TAFE advised that 2017 had the most reliable state-wide data and this contributed to the decision to use it as the baseline.

During the audit, TAFE sought advice from NSW Treasury regarding whether a 2017 baseline was appropriate and NSW Treasury advised that it was. Regardless, TAFE did not receive this advice prior to writing the CBA and did not put commentary around this in the CBA. This would also not be sufficient for fulfilling the requirements of the Guidelines.

The CBA did not comply with sections 6.3 and 6.4 of the Guidelines. The CBA briefly considered the impact of spending the campaign budget directly on new training courses, however there was no sustained analysis of this option. TAFE staff advised that there are no realistic alternatives to advertising for achieving the campaign's objectives. However we did not see analysis to support this conclusion in documents provided to us. 

The campaign CBA could have better demonstrated cost effectiveness if it considered alternative media mixes as outlined in NSW Treasury's 'Cost Benefit Analysis Framework for Government Advertising and Information Campaigns'. This would also have been consistent with the Handbook.

TAFE made one inaccurate claim in its advertising and overstated a second

The Guidelines set out rules regarding the content of a government advertising campaign. Exhibit 6 sets out one of the principles with which agencies must comply.

TAFE made one inaccurate claim in its advertising and overstated a second.

In some campaign creative material, TAFE claimed that 78 per cent of its own graduates are employed after training (Exhibit 15 in Appendix 2). According to the National Centre for Vocational Education Research, 78 per cent of New South Wales Vocational Education and Training (VET) graduates (i.e. from all training providers) are employed after training. The result for TAFE graduates is 70.4 per cent.

One of the campaign's television advertisements refers to TAFE as ‘Australia's most reputable education provider’. This statement referred to a survey of current TAFE students who were asked where they would consider studying in future: TAFE, University or a private college. The current TAFE students selected TAFE by a large margin. The limited scope of TAFE's student survey and its results do not support the claim that it is ‘Australia's most reputable education provider’.

DPC did not consistently communicate the transitional arrangements for the Brand Guidelines and as such much of TAFE's creative material did not comply at campaign launch

On 7 August 2017, the government released the NSW Government Brand Guidelines (Brand Guidelines), setting out how agencies use the NSW Government logo. The Brand Guidelines replaced the Branding Style Guide which had been in place since September 2015. Some agencies were exempt from using the Branding Style Guide and the introduction of the new Brand Guidelines required these agencies to apply for a new exemption.

TAFE had recently commenced the peer review process for this campaign when the Brand Guidelines were released. TAFE was exempt from the requirements of the Branding Style Guide and as such the material which TAFE was planning to re-use in the new campaign did not contain the NSW Government logo.

Communication about how long agencies had to make themselves compliant with the Brand Guidelines was unclear. On 11 August 2017, the Chair of the Cabinet Standing Committee on Communication and Government Advertising (the Committee) sent a letter to the Secretary of the Department of Industry informing him that the Department must update all its material to be compliant with the Brand Guidelines ‘as soon as practicable within an 18-month transition period’. The Department of Industry advised TAFE that new advertising would need to be immediately compliant, however it was not clear if this included materials which agencies were re-using from previous campaigns. DPC advised the audit team that it expected re-used materials to be compliant when agencies launched new campaigns. DPC provided this advice to some agencies but did not communicate it more broadly. We could not source evidence that DPC provided this advice to TAFE.

DPC ran workshops to explain the transitional arrangements in September 2017 for the changes in the Brand Guidelines, however these did not specifically address the transitional timeframes for new advertising campaigns.

The Department of Industry, on behalf of TAFE, applied to the Committee for approval to co-brand the TAFE logo with the NSW Government logo. This was approved in October 2017. The requirements for co-branding are in Exhibit 7.

Appendix one - Responses from agencies 

Appendix two - About the campaigns

Appendix three - About the audit

Appendix four - Performance auditing

Parliamentary reference - Report number #311 - released 18 December 2018

The urban renewal projects on former railway land in the Newcastle city centre are well targeted to support the objectives of the Newcastle Urban Transformation and Transport Program (the Program), according to a report released today by the Auditor-General for New South Wales, Margaret Crawford. The planned uses of the former railway land achieve a balance between the economic and social objectives of the Program at a reasonable cost to the government. However, the evidence that the cost of the light rail will be justified by its contribution to the Program is not convincing.

The Newcastle Urban Transformation and Transport Program (the Program) is an urban renewal and transport program in the Newcastle city centre. The Hunter and Central Coast Development Corporation (HCCDC) has led the Program since 2017. UrbanGrowth NSW led the Program from 2014 until 2017. Transport for NSW has been responsible for delivering the transport parts of the Program since the Program commenced. All references to HCCDC in this report relate to both HCCDC and its predecessor, the Hunter Development Corporation. All references to UrbanGrowth NSW in this report relate only to its Newcastle office from 2014 to 2017.

This audit had two objectives:

1. To assess the economy of the approach chosen to achieve the objectives of the Program.
2. To assess the effectiveness of the consultation and oversight of the Program.

We addressed the audit objectives by answering the following questions:

a) Was the decision to build light rail an economical option for achieving Program objectives?
b) Has the best value been obtained for the use of the former railway land?
c) Was good practice used in consultation on key Program decisions?
d) Did governance arrangements support delivery of the program?

Appendix one - Response from agencies    

Appendix two - About the audit

Appendix three - Performance auditing

Parliamentary reference - Report number #310 - released 12 December 2018