What the report is about

The Government Advertising Act 2011 requires the Auditor General to conduct a performance audit on government advertising activities each financial year.

This audit looked at whether three campaigns run by Destination NSW (DNSW) during 2020–21 were carried out in an effective, economical and efficient manner:

• Love Sydney (comprising two sub campaigns being ‘Sydney - Love It Like You Mean It’ and ‘Get Your Sydney On’)
• Love NSW
• Road Trips. 

What we found

DNSW complied with section 6 of the Government Advertising Act 2011 (the Act), prohibiting political content.

The Act requires the head of an agency to sign a compliance certificate that certifies that the campaign complies with the Act and is an efficient and cost effective means of achieving its public purpose. 

When the Acting Chief Executive of DNSW signed DNSW’s compliance certificate, evidence to support this certification was not available.

The Act requires a peer review and cost benefit analysis for campaigns over $1.0 million. DNSW did not complete the peer review or cost-benefit analysis for the audited advertising campaigns before they had concluded. 

The Department of Customer Service (DCS), which manages the peer review process, did not escalate the issue of the outstanding peer review documentation to senior DNSW staff. 

DNSW did not set targets for all measures established for the campaigns. This limits the ability to assess their effectiveness.

The impact of the COVID-19 pandemic likely contributed to the campaigns not meeting a substantial proportion of established outcome and impact targets.

None of the audited campaigns met the minimum requirement of 7.5 per cent for the allocation of the media budget for communications with Culturally and Linguistically Diverse and Aboriginal audiences.

What we recommended

DNSW should:

• implement processes for planning and delivering advertising campaigns delivered in urgent circumstances to bring them in line with NSW Government practice
• ensure that it establishes measurements and targets for outcomes and impacts of its advertising campaigns consistent with NSW Government evaluation frameworks and guidance.

The Department of Customer Service should:

• establish a policy and procedure for ensuring that campaign documentation is completed in a timely manner in the case of urgent campaigns, including establishing expectations around timeframes for the completion of peer review
• establish a procedure for escalating issues of outstanding documentation to ensure that the peer review is completed in line with reasonable expectations and timeframes.

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). This audit examined three campaigns run by Destination NSW during the 2020–21 financial year:

• Love Sydney (comprising two sub-campaigns being ‘Sydney - Love It Like You Mean It’ and ‘Get Your Sydney On’), focussing on increasing visitor activity in Sydney
• Love NSW, focussing on increasing visitor activity in regional New South Wales
• Road Trips, focussing on encouraging visitor activity on iconic road trips in regional New South Wales.

Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of Parliament or a candidate nominated for election to Parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.

The Act and associated regulations and the Guidelines also establish an accountability and compliance framework around the investment in advertising by NSW Government agencies.

The government's operating circumstances at the commencement of the 2020–21 financial year were highly challenging, with the 2019–20 bushfires being followed by the COVID-19 pandemic. This created new demands across a range of government services, and without any clear view on the severity of the pandemic and when it would end. This was the case for Destination NSW, which had to plan for its advertising activities in the context of an uncertain future for national border closures (impacting international in-bound travel) and lockdowns across Australia, including in New South Wales (impacting domestic travel). Further, the sudden nature of outbreaks and lockdowns meant that Destination NSW often was required to change the targeting of its campaigns and, in some situations, had to cease particular advertising activities until specific lockdowns had ended.

Campaign materials we reviewed did not contain political content

The audit team reviewed campaign materials developed as part of each of the paid advertising campaigns including radio transcripts, digital videos and display. See Appendix two for examples of campaign materials for this campaign.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material reviewed.

All reviewed campaigns were for purposes permitted by section 1.2 of the Guidelines

Section 4 of the Act states that government advertising campaigns are 'the dissemination to members of the public of information about a government program, policy or initiative, or about any public health or safety or other matter'. To support this, section 1.2 of the NSW Government Advertising Guidelines states that government advertising campaigns may only be used to achieve certain objectives. One of these objectives is to encourage changed behaviours or attitudes that will lead to improved public health and safety or quality of life.

The audit team considers that each of the reviewed advertising campaigns was consistent with this objective. This reflects the intent of each of the campaigns to increase economic activity driven by tourism activity in New South Wales, that contributes to improved quality of life for New South Wales residents.

The Acting Chief Executive signed Destination NSW's compliance certificate without supporting evidence

The Acting Chief Executive of Destination NSW signed a single compliance certificate for all Destination NSW campaigns for 2020–21 (including the three campaigns that are considered by this audit) on 28 February 2020. Evidence was not available at this date to support the statements included in the compliance certificate for the campaigns that were considered by this audit.

The compliance certificate is required by section 8 of the Act and states that the head of the agency confirms that a proposed government advertising campaign:

• complies with the Act, the regulations and the Guidelines, and
• contains accurate information, and
• is necessary to achieve a public purpose and is supported by analysis and research, and
• is an efficient and cost-effective means of achieving that public purpose.

At the time of signing the certificate in February 2020, Destination NSW had not conceived, designed or planned any of the campaigns that are considered by this audit, nor had it developed the relevant supporting information that would enable the agency to support these statements. As noted above, peer review had not commenced prior to this date. Further, Destination NSW had not completed a cost-benefit analysis or equivalent analysis.

Without any form of cost-benefit analysis or other evaluation for any of the campaigns prior to the date of signing of the compliance certificate, the Acting Chief Executive had no evidence that could support the certification that the campaigns were 'an efficient and effective means of achieving the public purpose'. The absence of peer review or a cost-benefit analysis also means that the Acting Chief Executive could not certify that the campaigns complied with the Act, the regulations or the Guidelines, nor that the campaign was supported by analysis and research.

Destination NSW did not complete peer reviews for the advertising campaigns before they ended, limiting assurance over campaign effectiveness, efficiency and economy

As all the campaigns subject to this audit were valued at over $250,000, each campaign was required to undergo peer review. The peer review is an independent review of the need for the proposed advertising campaign, the creative and media strategy (including objectives and target audiences) and how the agency will manage the campaign. Ordinarily, a peer review would be completed prior to a campaign commencing, however section 7(4) of the Act permits agencies to carry out a peer review after the advertising campaign commences 'if the head of the government agency concerned is satisfied that the campaign relates to an urgent public health or safety matter or is required in other urgent circumstances'.

DCS supported Destination NSW's assessment that these were urgent campaigns and that it would accept consideration of peer review components in parallel with the roll-out of the advertising campaigns, given the urgency of the need to generate economic activity, initially after the 2019–20 bushfires and then after the challenging circumstances brought on by the COVID-19 pandemic. This is in line with section 7(4) of the Act.

Destination NSW presented and obtained clearance on creative materials and media planning on a timely basis for two of the three campaigns (but not for the Road Trips campaign), which would ordinarily form part of peer review. However, for all campaigns, the peer reviews were not completed or signed off by DCS prior to the completion of advertising campaigns. In particular, Destination NSW did not submit material related to the accountability for campaign effectiveness, including the campaign objectives and measures before the end of the campaigns.

The absence of peer review of much of the material prior to completion of the campaigns reduces the ability of the agency and government to be confident that the advertising expenditure was consistent with NSW Government requirements, or represented efficient, effective and economical use of funds.

Destination NSW noted that section 7(4) of the Act allows the peer review to be completed after the commencement of a campaign in urgent circumstances but places no requirement on it to be completed before the end of the campaign. The audit has determined that for the peer review to meet its intended purpose, being to inform the design and delivery of the advertising campaign, it needs to be completed prior to the end of the campaign, even in urgent circumstances. DCS has supported this intent of the framework.

By the end of September 2021, DCS advised Destination NSW that it would not consider any further material for peer review related to the 2020–21 advertising campaigns. At this time, DCS closed the peer review for the Love NSW and Road Trips campaigns and assessed them as incomplete. DCS assessed the Love Sydney peer review as complete, despite noting that the campaign evaluation was not complete and with no details or confirmation of meeting culturally and linguistically diverse (CALD) advertising requirements, including for Aboriginal communities.

DCS did not escalate the issue of outstanding peer review materials

DCS worked at officer level to remind Destination NSW that peer review material was outstanding during the year. While this is appropriate as an initial point of escalation, at no time was the issue of non-compliance escalated to higher levels of management. DCS also never sent formal correspondence requesting the materials needed to ensure the completion of peer review.

DCS does not have a process for ensuring the timely completion of peer review in situations where urgency exemptions are used. There is an opportunity to formalise this process to ensure that there are appropriate escalation points and to ensure that compliance obligations are fulfilled in future.

Destination NSW did not meet the minimum requirement for allocation of the media budget for communications with CALD and Aboriginal audiences

The NSW Government 'CALD and Aboriginal Advertising Policy' stipulates that at least 7.5 per cent of an advertising campaign media budget is to be spent on direct communications to multicultural and Aboriginal audiences. Spend may be on media or non-media communication activities (e.g. events, participation at cultural festivals, direct mail, competitions and websites).

Destination NSW spent only 1.6 per cent of its media spend on culturally and linguistically diverse specific media placement on the 'Sydney - Love It Like You Mean It' campaign and none of its media placement for the other audited campaigns. This level of expenditure is substantially below the requirement.

Destination NSW could not demonstrate how its campaign designs or media placements effectively supported the cultural needs and issues of culturally and linguistically diverse populations. In connection with the 'Sydney - Love It Like You Mean It' campaign, it was noted that timeframes and production issues limited the ability to incorporate culturally diverse individuals in imagery.

Destination NSW advised that it believes the application of a 7.5 per cent threshold for specific audiences is not an effective way to reach these audiences. Destination NSW advised that its advertising was targeted at audiences with a propensity to travel, which did not necessarily include culturally diverse audiences, and its media channel research influenced its decision not to target specific CALD-focussed media channels.

None of the above factors negate Destination NSW's responsibility to ensure that the 'CALD and Aboriginal Advertising Policy' requirements are met.

In addition, Destination NSW also noted a number of non-media activities that supported culturally and linguistically diverse audiences, including translations on the sydney.com website, capturing of culturally and linguistically diverse audiences in production shooting and the production of a range of other collateral for culturally and linguistically diverse audiences. Despite these non-media activities, which Destination NSW did not quantify, the requirement for minimum expenditure in the reviewed campaigns for CALD audiences was not met by Destination NSW.

Destination NSW advised that it believes that the 7.5 per cent requirement does not apply to advertising outside of New South Wales, which the 'Get Your Sydney On', Love NSW and Road Trips campaigns targeted in whole or in part. The 'CALD and Aboriginal Advertising Policy' does not specifically limit its application to advertising for New South Wales residents.

Appendix one – Response to Destination NSW

Appendix two – Response from agencies

Appendix three – About the campaigns

Appendix four – About the audit

Appendix five – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #360 - released (23 December 2021).

What the report is about

The term ‘machinery of government’ refers to the way government functions and responsibilities are organised.

The decision to make machinery of government changes is made by the Premier. Changes may be made for a range of reasons, including to support the policy and/or political objectives of the government of the day.

Larger machinery of government changes typically occur after an election or a change of Premier.

This report assessed how effectively the Department of Planning, Industry and Environment (DPIE) and the Department of Regional NSW (DRNSW) managed their 2019 and 2020 machinery of government changes, respectively. It also considered the role of the Department of Premier and Cabinet (DPC) and NSW Treasury in overseeing machinery of government changes.

What we found

The anticipated benefits of the changes were not articulated in sufficient detail and the achievement of benefits has not been monitored. The costs of the changes were not tracked or reported.

DPC and NSW Treasury provided principles to guide implementation but did not require departments to collect or report information about the benefits or costs of the changes.

The implementation of the machinery of government changes was completed within the set timeframes, and operations for the new departments commenced as scheduled.

Major implementation challenges included negotiation about the allocation of corporate support staff and the integration of complex corporate and ICT systems.

What we recommended

DPC and NSW Treasury should:

• consolidate existing guidance on machinery of government changes into a single document that is available to all departments and agencies
• provide guidance for departments and agencies to use when negotiating corporate services staff transfers as a part of machinery of government changes, including a standard rate for calculating corporate services requirements
• progress work to develop and implement common processes and systems for corporate services in order to support more efficient movement of staff between departments and agencies.

The term ‘machinery of government’ refers to the way government functions and responsibilities are allocated and structured across government departments and agencies. A machinery of government change is the reorganisation of these structures. This can involve establishing, merging or abolishing departments and agencies and transferring functions and responsibilities from one department or agency to another.

The decision to make machinery of government changes is made by the Premier. These changes may be made for a range of reasons, including to support the policy and/or political objectives of the government of the day. Machinery of government changes are formally set out in Administrative Arrangements Orders, which are prepared by the Department of Premier and Cabinet, as instructed by the Premier, and issued as legislative instruments under the Constitution Act 1902.

The heads of agencies subject to machinery of government changes are responsible for implementing them. For more complex changes, central agencies are also involved in providing guidance and monitoring progress.

The NSW Government announced major machinery of government changes after the 2019 state government election. These changes took place between April and June 2019 and involved abolishing five departments (Industry; Planning and Environment; Family and Community Services; Justice; and Finance, Services and Innovation) and creating three new departments (Planning, Industry and Environment; Communities and Justice; and Customer Service). This also resulted in changes to the 'clusters' associated with departments. The NSW Government uses clusters to group certain agencies and entities with related departments for administrative and financial management. Clusters do not have legal status. Most other departments that were not abolished had some functions added or removed as a part of these machinery of government changes. For example, the functions relating to regional policy and service delivery in the Department of Premier and Cabinet were moved to the new Department of Planning, Industry and Environment.

Our Report on State Finances 2019, tabled in October 2019, outlined these changes and identified several issues that can arise from machinery of government changes if risks are not identified early and properly managed. These include: challenges measuring the costs and benefits of machinery of government changes; disruption to services due to unclear roles and responsibilities; and disruption to control environments due to staff, system and process changes.

In April 2020, the Department of Regional NSW was created in a separate machinery of government change. This involved moving functions and agencies related to regional policy and service delivery from the Department of Planning, Industry and Environment into a standalone department.

This audit assessed how effectively the Department of Planning, Industry and Environment (DPIE) and the Department of Regional NSW (DRNSW) managed their 2019 and 2020 machinery of government changes, respectively. It also considered the role of the Department of Premier and Cabinet and NSW Treasury in overseeing machinery of government changes. The audit investigated whether:

• DPIE and DRNSW have integrated new responsibilities and functions in an effective and timely manner
• DPIE and DRNSW can demonstrate the costs of the machinery of government changes
• The machinery of government changes have achieved or are achieving intended outcomes and benefits.

Appendix one – Response from agencies

Appendix two – About the audit

Appendix three – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #359 - released (17 December 2021).

A report released today by the Auditor-General for New South Wales, Margaret Crawford found that select advertising campaigns conducted by Service NSW and the NSW Rural Fire Service met most requirements of the Government Advertising Act, regulations, Guidelines and other laws. However, the audit found that Service NSW inappropriately used its post campaign evaluation to measure sentiment towards and confidence in the NSW Government.  

While agency analysis shows that the ‘Cost of Living’ (phases 2 and 3)  and ‘How Fireproof is Your Plan?’ campaigns achieved most of their objectives, the campaign objectives and targets set by both agencies were not sufficient to measure all aspects of campaign effectiveness. 

The report makes two recommendations to the Department of Customer Service. The first is to review its guidance to ensure agencies are not using post campaign evaluations to measure sentiment towards the government. The second, to review its guidance and the new process of peer review to ensure they support agencies to comply with the Act, the regulations and the Guidelines. 

The Government Advertising Act 2011 requires the Auditor General to conduct an annual performance audit of one or more government agencies to see whether their advertising activities were carried out in an effective, economical and efficient manner and in compliance with the Government Advertising Act 2011.
 

Read full report (PDF)

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). This audit examined two campaigns run during the 2018–19 and 2019–20 financial years respectively:

• the 'Cost of Living' campaign run by Service NSW (phases 2 and 3 delivered in 2018–19)
• the 'How Fireproof Is Your Plan?' (Fireproof) campaign run by NSW Rural Fire Service (year two of a three-year campaign delivered in 2019–20).

Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.

In 2018–19, Service NSW delivered phases 2 and 3 of the 'Cost of Living' campaign. The Cost of Living advertising campaign aimed to build awareness of the help available to ease the cost of living for people under financial pressure including awareness of specific rebates that can be claimed. As part of the Cost of Living program, Service NSW developed a webpage designed as a single portal to access more than 40 NSW Government savings, rebates and initiatives (which originated from over 12 different agencies). It also launched the Cost of Living service which includes face to face meetings and phone interviews to help people claim rebates from the NSW Government. Phase 2 of the campaign ran from September 2018 to August 2019. Phase 3 of the campaign ran from January 2019 to July 2019. The budgets for phases 2 and 3 were $4.127 million and $934,800 respectively. See Appendix two for more details on this campaign.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign including radio transcripts, digital videos and display. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. See Appendix two for examples of campaign materials for this campaign.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

Post-campaign evaluations measured sentiment towards and confidence in the NSW Government

The post-campaign evaluation for phases 2 and 3 measured levels of confidence with the statement ‘the NSW Government has your best interests at heart’, despite the fact this was not a stated objective of the campaign. This is not an appropriate use of the post-campaign evaluation, which should measure the success of the campaign against its stated objectives. The post-campaign evaluation for phase 3 found that exposure to the campaign improved sentiment towards the government amongst those who did not have confidence in the NSW Government.

Service NSW advised that it was important to measure the sentiment of the advertising including the wording 'best interests' as it did not want the whole of government brand to be detrimental to customer engagement with applying for the rebates.

Following phase 2, Service NSW conducted analysis of media sentiment using the key words 'cost of living' and the names of the Premier, Treasurer and Minister for Customer Service. The analysis presented the level of positive, negative and neutral media sentiment. The Government Advertising Guidelines 2012 list the purposes that government advertising may serve which do not include improving the perception of the government. The inclusion of this analysis in Service NSW's post-campaign evaluation creates a risk that the results may be used for party political purposes.

Section 10 of the Act restricts agencies from carrying out a campaign after 26 January in the calendar year before the Legislative Assembly is due to expire and before the election for the Legislative Assembly in that year. Service NSW authorised a media agency to book media in line with the media plans for the campaign. The media plans for the campaign show that Service NSW did not authorise or plan to run any advertisements between 27 January 2019 and 23 March 2019.

Service NSW did not set targets for all rebates advertised in phase 2

Service NSW did not set targets for four of the seven rebates that were advertised as part of phase 2 of the campaign. These rebates were the Family Energy Rebate, Appliance Replacement Offer, National Parks Concession Offer and the Pensioner Travel Voucher. As a result, it was unable to evaluate whether the advertisements for these rebates were effective. Service NSW advised that at the time the campaign went to peer review, when campaign objectives are set, it did not know which rebates would be included in the advertisements.

Service NSW stated in its submission to the Department of Premier and Cabinet that it may change the creative content for phase 2 as it announced new initiatives and rebates. The peer review process should have ensured that Service NSW set targets for any additional rebates or savings it intended to advertise before that advertising commenced to ensure a strategic approach to the campaigns that clearly demonstrated anticipated benefits were in place.

The post-campaign evaluation for phase 2 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for some rebates advertised as part of phase 2 of the campaign. The objectives, targets and results for phase 2 are shown in Exhibit 5. In phase 2, Service NSW established baseline data on levels of awareness of government rebates during the peer review process. The baseline level of awareness for government rebates was 44 per cent. The level of awareness for specific rebates was 46 per cent for the Compulsory Third Party (CTP) green slip refund, and 21 per cent for both Active Kids and Toll Relief.

Post-campaign evaluation reports for phase 2 show that the campaign met its objective to raise awareness of NSW Government rebates, achieving a 16 per cent increase in awareness from 44 per cent to 51 per cent. The campaign did not meet its target to increase awareness of the CTP green slip refund by ten per cent.

Service NSW did not report the results of the uptake of the CTP green slip refund, Active Kids and Toll Relief in its post campaign effectiveness report submitted to the Department of Premier and Cabinet. However, other post-campaign evaluation documentation, which Service NSW advise was submitted to the Department of Premier and Cabinet, show that these targets were met.

Service NSW did not report to the Department of Premier and Cabinet on whether it achieved the target of a ten per cent increase of average monthly visits to the Cost of Living webpage. Service NSW reported that it had achieved an average of 11,753 visitors to the webpage per day during the campaign. These average daily results indicate that the target was met.

Exhibit 5: Phase 2 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. a) Increase awareness of rebates from the NSW Government by ten per cent.	Image
b) Increase average monthly visits to the Cost of Living webpage by ten per cent.	Image
2. Increase awareness of rebates and savings by ten per cent for:
CTP green slip refund	Image
Active Kids	Image
Toll Relief.	Image
3. Increase awareness that NSW Government initiatives relating to the cost of living are available via Service NSW by ten per cent.	Image
4. Increase the uptake of rebates and savings for the CTP green slip refund, Active Kids and Toll Relief by ten per cent.	Image

Key

Image

Yes

Image

Not Fully

The post-campaign evaluation for phase 3 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for the two rebates advertised as part of phase 3 of the campaign. The objectives, targets and results for phase 3 are shown in Exhibit 6.

In phase 3, Service NSW established baseline data on levels of awareness during the peer review process. The baseline level of awareness for government rebates was 44 per cent. This is the same baseline that was used to measure performance for phase 2 of the campaign. Service NSW did not set baselines for awareness and uptake of Energy Switch and Creative Kids as these were new services.

Post-campaign evaluation reports for phase 3 show that the campaign met its objective to raise awareness of NSW Government rebates by ten per cent, achieving a 30 per cent increase in awareness from 44 per cent to 57 per cent. The overall increase in message take-out was met with 43 per cent agreeing with the message that the NSW Government is taking steps to ease the cost of living. The campaign achieved awareness and uptake targets for the specific rebates included in phase 3, except for awareness of Creative Kids which achieved 28 per cent awareness, falling short of the 30 per cent awareness target.

Exhibit 6: Phase 3 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. Increase message takeout that ‘The NSW Government is taking steps to help ease the cost of living in NSW’ by ten per cent for those who can recall the campaign.
2. Increase awareness that the NSW Government has a range of rebates and savings by ten per cent.
3. Generate awareness with NSW residents aged 18+ of:
Energy Switch (15 per cent awareness)
Creative Kids (30 per cent awareness).
4. Create uptake of Energy Switch and Creative Kids (8,356 clicks on the Energy Switch website and 107,938 Creative Kids vouchers downloaded with 70 per cent conversion).

Key

Yes

Not Fully

The timing of campaign phases meant that it was difficult for Service NSW to evaluate each distinct campaign phase and reduced opportunities to incorporate learnings from previous phases

Service NSW commenced planning for phase 2 of the campaign while phase 1 was still underway. This limited the opportunity for Service NSW to incorporate learnings from phase 1 into phase 2. There was some overlap in the timing of phase 2 and the start of phase 3 of the campaign, making it difficult to evaluate the effectiveness of each distinct campaign phase. Both phases 2 and 3 had the same high-level outcome objective to raise awareness of rebates by ten per cent. The baseline measures that were used to evaluate performance for phase 3 were the same as those used to evaluate phase 2. As a result, Service NSW was not able to separately evaluate these two phases of the campaign. This is important given the budgets for phases 2 and 3 were $4.127million and $934,800 respectively.

Service NSW allocated 7.5 per cent of its media budget to communications with culturally and linguistically diverse (CALD) and Aboriginal audiences

The NSW Government CALD and Aboriginal Advertising Policy requires that agencies spend at least 7.5 per cent of an advertising campaign media budget on direct communications with CALD and Aboriginal audiences. Service NSW authorised a media company to book media in line with the media plans for the campaign. The media plans for phases 2 and 3 of the campaign indicate that Service NSW met this requirement, with 7.5 per cent of the budget allocated to these audiences in phase 2 and 10.4 per cent in phase 3.

The post campaign evaluation for phases 1 and 2 of the Cost of Living campaign contained a recommendation to look at other opportunities to reach CALD audiences. Effective communication with CALD audiences was particularly important in phase 3 of the campaign, where they made up 30 per cent of the target audience for the Creative Kids advertisement. The post-campaign analysis for phase 3 showed that the campaign performed well with some, but not all CALD audiences. The post-campaign analysis also showed low awareness and uptake with Aboriginal audiences. Pre-campaign focus groups in phase 3 found Aboriginal audiences had a negative reaction to the campaign tag line ‘NSW Government is helping with the cost of living’ however this tagline was still used in some advertisements in phase 3.

The cost-benefit analysis (CBA) for phase 2 did not accurately assess the benefits of the campaign and did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. The CBA conducted by Service NSW for phase 2 includes $8 million in benefits attributed to the advertisements for the Energy Switch tool and $6.9 million in benefits attributed to the advertisements for Creative Kids vouchers. These benefits should not have been included in the CBA for phase 2 as they were not included in this phase of the campaign. The CBA did not estimate the benefits of some other rebates and savings advertised in phase 2 of the campaign. This means that the CBA did not accurately assess the benefits of the campaign. Service NSW advised that at the time the CBA was developed it had not selected the rebates to be included in the campaign.

The Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits. The CBA developed as part of phase 2 identified using existing NSW Government communication channels as an alternative to advertising but did not assess the costs and benefits of this alternative.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Service NSW negotiated with a single creative agency in phase 2, making it difficult to demonstrate value for money

Agencies are required to obtain three quotes when procuring a creative agency on the prequalification scheme if the estimated cost of the creative content is greater than $150,000. In phase 2 of the campaign, Service NSW extended the contract with the creative agency used for phase 1 of the campaign and did not obtain three quotes despite the cost of the creative content for phase 2 being $731,480. The requirement to obtain three quotes was met in phase 1 when initially selecting this creative agency.

Service NSWs procurement policy details that direct negotiation may be appropriate where there is a compelling reason to renew or rollover a contract beyond temporal or convenience reasons or in the cases of a genuine emergency. In its briefing to the Chief Executive, Service NSW stated that this contract extension was sought due to the time-sensitive nature of the project and that if work was delayed by a tender process, Service NSW may not be able to meet marketing milestones and this could result in limited customer uptake. This reason is not a genuine emergency and is not compelling as it does not explain what consequences would occur if it did not meet the marketing milestones or if there was limited customer uptake.

Service NSW's procurement policy also states that under no circumstances must Service NSW employees enter into discussions with a supplier until the delegate has formally made their decision to enter into direct negotiation. Service NSW briefed the Chief Executive of Service NSW in relation to extending the contract on 5 September 2018. The briefing states that the creative agency had already begun developing creative content for phase 2 and Service NSW had already received quotes from the creative provider for the proposed work prior to 5 September 2018. Procurement sign-offs were not completed until 7 September 2018. The engagement of the creative provider prior to appropriate approvals was contrary to Service NSWs procurement policy.

The economy of the campaign may have been limited by not meeting the procurement requirements in phase 2. It is possible that the creative provider may have offered a more competitive rate if it was aware that Service NSW was seeking quotes from other creative providers. Additionally, it is possible that another creative provider could have provided better value for money.

In phase 3 of the campaign, the estimated cost of the creative exceeded $150,000 however Service NSW chose to contract two different creative agencies, and the cost for each agency fell below the threshold to obtain three quotes. Agencies are permitted to obtain one quote when using a creative provider on the prequalification scheme if the cost is between $50,000 to $150,000. Service NSW advised that it contracted two creative providers as two different project teams were responsible for the rebates, each with separate marketing budgets.

Service NSW allowed sufficient time for cost-efficient media placement

During the peer review process, the Department of Premier and Cabinet advised agencies about the time they should allow to ensure cost-efficient media placement. For example, the Department of Premier and Cabinet advised that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance.

Service NSW allowed sufficient time between the completion of the peer review process and the commencement of the first advertising. Service NSW signed the agreement with the approved Media Agency Services provider with sufficient time to achieve cost-efficient media placement for all types of media used in this campaign.

The campaign may have been misleading for some people who were not eligible for rebates

Advertisements we reviewed focused on the amount of savings that could be obtained from rebates, for example ‘Save up to $285’, and ended with a statement ‘To save, visit service.nsw.gov.au. This directed viewers to the Cost of Living website which contains eligibility information. However, the advertisements in phases 2 and 3 we reviewed did not contain any details on the eligibility for these rebates and not all advertisements stated that eligibility criteria apply. Service NSW advised that the eligibility criteria for each rebate is extensive and that it was not possible to include this in the creative material.

Post-campaign evaluations in phase 3 recommended that advertisements for Creative Kids should indicate eligibility (e.g. age criteria) as statements on savings have the potential to be misleading when not all viewers will be eligible for rebates. Social media analysis conducted following phase 2 showed ineligibility or inability to claim rebates or refunds caused anger for some respondents.

Some advertisements in phase 2 stated ‘we've got something for everyone’. However, as rebates were subject to eligibility criteria, it is possible that some residents in NSW would not be eligible for any rebates as part of the Cost of Living initiative. As such, this statement has the potential to be misleading.

The campaign included statements that underestimated the savings that some customers could obtain

The Guidelines require accuracy in the presentation of all facts, statistics, comparisons and other arguments. The Guidelines also require that all claims of fact included in government advertising campaigns must be able to be substantiated.

In phase 2, the possible savings customers could obtain for two rebates or savings exceeded the amounts stated in the advertising campaign. Exhibit 7 shows some advertisements in phase 2 which stated, ‘My Green Slip Saving Save up to $60’. However, the State Insurance Regulatory Authority website shows that savings for some types of motor vehicles under the 2017 CTP scheme exceed $60. The State Insurance Regulatory Authority website states that the average saving under this scheme has been $129. Service NSW advised that these advertisements were designed for regional markets and that it used different advertisements for metropolitan areas which contained different amounts of savings.

Some advertisements in phase 2 stated, ‘My Toll Relief save up to $700’. The Service NSW website states that drivers can obtain free vehicle registration if they have spent $1,352 or more in tolls in the previous financial year. The cost of registration for some vehicles exceeds $700. This means the savings detailed in the advertisement were lower than what some customers could actually save.

NSW Rural Fire Service conducted the 'How FireProof Is Your Plan?' (Fireproof) campaign. The Fireproof campaign is a three-year campaign which ran in 2018–19 (year one), 2019–20 (year two) and is planned for 2020–21 (year three). This audit examined year two of the campaign (2019–20).

The Fireproof campaign is a public safety campaign encouraging people to plan and prepare for bush fires across the summer period. The campaign aims to improve the quality of bush fire planning and preparation in the community and decrease the impact of fires on the community when they occur.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign for example radio advertisements, television commercials and digital displays. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. Examples of campaign materials are shown in Appendix two.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

NSW Rural Fire Service did not set targets for the second year of the campaign

The second year of the Fireproof campaign (2019–20) had the same objectives as the first year of the campaign (2018–19), however no specific targets were set for the second year. The advertising submission for the first year of the campaign (2018–19) details the targets for each objective as an increase of ten per cent against the baseline data to be achieved by March 2021, at the end of the three-year campaign.

The second year of the Fireproof campaign (2019–20) was one of the first campaigns approved under the new budget and peer review processes introduced by the Department of Customer Service in 2019–20. The new process for peer review introduced a new template for campaign submissions. The former template for campaign submissions contained more prompts for agencies to ensure the submission contained sufficient detail of campaign objectives, baseline measures, targets, dates for measurement and detail on how they would measure objectives. Despite this, the peer review process should have identified that NSW Rural Fire Service did not set targets for the second year of the campaign.

The 2016 Guidelines for Implementing NSW Government Evaluation Framework for Advertising and Communications requires campaign objectives to be SMART (specific, measurable, achievable, realistic and timed). NSW Rural Fire Service did not meet this requirement for year two of the Fireproof campaign.

Post-campaign evaluations showed increases against four out of five objectives, however there were no specific targets

NSW Rural Fire Service set three campaign objectives at the time it submitted the second year of the campaign (2019–20) to the Department of Customer Service for peer review. However, the post-campaign effectiveness report submitted to the Department of Customer Service measured campaign effectiveness against five campaign objectives. The objectives in the post-campaign effectiveness report were the same objectives set for the first year of the campaign, which is appropriate as this was a repeat campaign.

NSW Rural Fire Service achieved increases against four of their five objectives. However, as noted above there were no specific targets (such as percentage increases) against which performance of the 2019–20 campaign could be measured. Despite this, at the end of the second year, the Fireproof campaign had already achieved some of the targets that NSW Rural Fire Service had set for the end of the third year of the campaign. The post-campaign research showed that both audience recall and exposure to the campaign increased significantly from the prior year. The campaign objectives and results are shown in Exhibit 8.

For those people who already have a bush fire plan, the campaign aimed to increase the number of those plans which have included two or more elements from the Guide to Making a Bush Fire Survival Plan. Elements from the Guide to Making A Bush Fire Survival Plan include actions such as deciding what to take with you if you leave, ensuring you have the right equipment for defending your home and allocating responsibilities to members of a household. The post-campaign evaluation showed that the campaign did not achieve an increase against this objective for people who planned to stay and defend their property rather than leave.

Exhibit 8: Campaign objectives and results

Campaign objectives	Does the post-campaign evaluation show increases against the objective?
1. Continue to increase the number of people that have discussed and/or written a plan with regards to what they will do in the event of a fire.
2. Of those who indicate they have a plan, increase the number of people who have included two or more elements from the Guide to Making a Bush Fire Survival Plan:
for those who plan to leave
for those who plan to stay and defend.
3. Increase the frequency in completing preparation activities around a person’s property.
4. Increase the number of people who correctly assess it is their responsibility to complete preparation activities and enact their plan without direct intervention from emergency services.
5. Visits to MyFirePlan website.

Key

Yes

No

NSW Rural Fire Service achieved cost efficiencies by reusing creative content developed in the first year of the campaign

Total creative and production costs incurred in year one of the campaign were $1.08 million. Rather than commissioning new creative materials, NSW Rural Fire Service re-used the same creative content in year two of the campaign. NSW Rural Fire Service incurred $100,000 in creative and production costs in year two of the campaign and achieved cost-efficiencies by reusing the same creative developed in the prior year.

NSW Rural Fire Service allowed sufficient time for cost-efficient media placement and received free media placements

The Department of Customer Service advises agencies to work with media contacts to book media in advance to ensure a cost-efficient placement. Prior to 2019–20, the Department of Premier and Cabinet provided suggested timeframes for agencies to book media as part of the peer review process. For example, it advised agencies to book television six to 12 weeks in advance and book radio advertising two to eight weeks in advance. NSW Rural Fire Service allowed sufficient time for a cost-efficient media placement.

NSW Rural Fire Service received $4 million of free advertising time and space donated by media companies due to the extent and impact of the 2019–20 fire season.

The cost benefit analysis (CBA) did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. As part of the CBA, the Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits.

The CBA for the Fireproof campaign (year two) notes that the proposed campaign is one component of a broader community engagement strategy which has been developed over time and is based on research and evaluation. The CBA considers two options to achieve the objectives of the campaign. The first option is community engagement activities without an advertising campaign and the second option is community engagement activities alongside an advertising campaign. The CBA does not identify and assess the costs and benefits of both of the options in order to assess the most cost-efficient option.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Appendix one – Responses from agencies

Appendix two – About the campaigns

Appendix three – About the audit

Appendix four – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #342 - released 19 November 2020

The Auditor-General for New South Wales, Margaret Crawford, released a report today examining whether the Department of Communities and Justice had effective governance and partnership arrangements in place to deliver ‘Their Futures Matter’.

Their Futures Matter was intended to place vulnerable children and families at the heart of services, and direct investment to where funding and programs deliver the greatest social and economic benefits. It was a four-year whole-of-government reform in response to the 2015 Tune Review of out-of-home care.

The Auditor-General found that while important foundations were put in place, and new programs trialled, the key objective to establish an evidence-based whole-of-government early intervention approach for vulnerable children and families in NSW was not achieved.

Governance and cross-agency partnership arrangements to deliver Their Futures Matter were found to be ineffective. 'Their Futures Matter lacked mechanisms to secure cross portfolio buy‑in and did not have authority to drive reprioritisation of government investment', the Auditor-General said.

At the reform’s close, the majority of around $380 million in investment funding remains tied to existing agency programs, with limited evidence of their comparative effectiveness or alignment with Their Futures Matter policy objectives. The reform concluded on 30 June 2020 without a strategy or plan in place to achieve its intent.

The Auditor-General made four recommendations to the Department of Communities and Justice, aimed at improving implementation of outstanding objectives, revising governance arrangements, and utilising the new human services data set to address the intent of the reform. However, these recommendations respond only in part to the findings of the audit.

According to the Auditor-General, ‘Cross-portfolio leadership and action is required to ensure a whole-of-government response to delivering the objectives of Their Futures Matter to improve outcomes for vulnerable children, young people and their families in New South Wales.’

Read full report (PDF)

In 2016, the NSW Government launched 'Their Futures Matter' (TFM) - a whole-of-government reform aimed at delivering improved outcomes for vulnerable children, young people and their families. TFM was the government's key response to the 2015 Independent Review of Out of Home Care in New South Wales (known as 'the Tune Review').

The Tune Review found that, despite previous child protection reforms, the out of home care system was ineffective and unsustainable. It highlighted that the system was not client-centred and was failing to improve the long-term outcomes for vulnerable children and families. The review found that the greatest proportion of relevant expenditure was made in out of home care service delivery rather than in evidence-based early intervention strategies to support children and families when vulnerabilities first become evident to government services (such as missed school days or presentations to health services).

The then Department of Family and Community Services (FACS) designed the TFM reform initiatives, in consultation with central and human services agencies. A cross-agency board, senior officers group, and a new unit in the FACS cluster were established to drive the implementation of TFM. In the 2016–17 Budget, the government allocated $190 million over four years (2016–17 to 2019–20) to the reform. This resourced the design and commissioning of evidence-based pilots, data analytics work, staffing for the implementation unit and secretariat support for the board and cross-agency collaboration.

As part of the TFM reform, the Department of Premier and Cabinet, NSW Treasury and partnering agencies (NSW Health, Department of Education and Department of Justice) identified various existing programs that targeted vulnerable children and families (such as the preceding whole-of-government ‘Keep Them Safe’ reform coming to an end in June 2020). Funding for these programs, totalling $381 million in 2019–20, was combined to form a nominal ‘investment pool’. The government intended that the TFM Implementation Board would use this pool to direct and prioritise resource allocation to evidence-based interventions for vulnerable children and families in NSW.

This audit assessed whether TFM had effective governance and partnership arrangements in place to enable an evidence-based early intervention investment approach for vulnerable children and families in NSW. We addressed the audit objective with the following audit questions:

• Was the TFM reform driven by effective governance arrangements?
• Was the TFM reform supported by effective cross-agency collaboration?
• Has the TFM reform generated an evidence base to inform a cross-agency investment approach in the future?

The audit did not seek to assess the outcomes for children, young people and families achieved by TFM programs and projects.

Their Futures Matter (TFM) is a whole-of-government reform to deliver improved outcomes for vulnerable children, young people and their families.

Supported by a cross-agency TFM Board, and the TFM Unit in the then Department of Family and Community Services (FACS), the reform aimed to develop whole-of-government evidence-based early intervention investment approaches for vulnerable children and families in NSW.

Governance refers to the structures, systems and practices that an organisation has in place to:

• assign decision-making authorities and establish the organisation's strategic direction
• oversee the delivery of its services, the implementation of its policies, and the monitoring and mitigation of its key risks
• report on its performance in achieving intended results, and drive ongoing improvements.

We examined whether the TFM reform was driven by effective governance arrangements and cross-agency collaboration.

The reform agenda and timeframe set down for Their Futures Matter (TFM) were ambitious. This chapter assesses whether the TFM Board and TFM Unit had the capability, capacity and clout within government to deliver the reform agenda.

Creating a robust evidence base was important for Their Futures Matter, in order to:

• identify effective intervention strategies to improve supports and outcomes for vulnerable children and families
• make efficient use of taxpayer money to assist the maximum number of vulnerable children and families
• inform the investment-based approach for future funding allocation.

This chapter assesses whether the TFM reform has developed an evidence base to inform cross-agency investment decisions.

Appendix one – Response from agency

Appendix two – TFM governance entities

Appendix three – TFM Human Services Data Set

Appendix four – TFM pilot programs

Appendix five – About the audit

Appendix six – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #337 - released 24 July 2020

This report outlines whether the Department of Customer Service (the department) has effective controls in place to ensure the integrity of data in the Births, Deaths and Marriages Register (the register), and to prevent unauthorised access and misuse.

The audit found that the department has processes in place to ensure that the information entered in the register is accurate and that any changes to it are validated. Although there are controls in place to prevent and detect unauthorised access to, and activity in the register, there were significant gaps in these controls. Addressing these gaps is necessary to ensure the integrity of information in the register.

The Auditor-General made nine recommendations to the department, aimed at strengthening controls to prevent and detect unauthorised access to, and activity in the register. These included increased monitoring of individuals who have access to the register and strengthening security controls around the databases that contain the information in the register.

The NSW Registry of Births Deaths and Marriages is responsible for maintaining registers of births, deaths and marriages in New South Wales as well as registering adoptions, changes of names, changes of sex and relationships. Maintaining the integrity of this information is important as it is used to confirm people’s identity and unauthorised access to it can lead to fraud or identity theft.

Read full report (PDF)

The NSW Registry of Births Deaths and Marriages (BD&M) is responsible for maintaining registers of births, deaths and marriages in New South Wales. BD&M is also responsible for registering adoptions, changes of name, changes of sex and relationships. These records are collectively referred to as 'the Register'. The Births, Deaths and Marriages Registration Act 1995 (the BD&M Act) makes the Registrar (the head of BD&M) responsible for maintaining the integrity of the Register and preventing fraud associated with the Register. Maintaining the integrity of the information held in the Register is important as it is used to confirm people's identity. Unauthorised access to, or misuse of the information in the Register can lead to fraud or identity theft. For these reasons it is important that there are sufficient controls in place to protect the information.

BD&M staff access, add to and amend the Register through the LifeLink application. While BD&M is part of the Department of Customer Service, the Department of Communities and Justice (DCJ) manages the databases that contain the Register and sit behind LifeLink and is responsible for the security of these databases.

This audit assessed whether BD&M has effective controls in place to ensure the integrity of data in the Births, Deaths and Marriages Register, and to prevent unauthorised access and misuse. It addressed the following:

• Are relevant process and IT controls in place and effective to ensure the integrity of data in the Register and the authenticity of records and documents?
• Are security controls in place and effective to prevent unauthorised access to, and modification of, data in the Register?

Appendix one – Response from agency

Appendix two – About the audit

Appendix three – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #330 - released 7 April 2020.

In October 2016, the NSW Government accepted an unsolicited proposal from IFM Investors and AustralianSuper to lease 50.4 per cent of Ausgrid for 99 years. The deal followed the Federal Government’s rejection of two bids from foreign investors, for national security reasons.

A performance audit of the lease of Ausgrid has found shortcomings in the unsolicited proposal process. Releasing the audit findings today, the Auditor-General for New South Wales, Margaret Crawford said ‘this transaction involved a $20 billion asset owned by the people of New South Wales. As such, it warranted strict adherence to established guidelines’.

Ausgrid is a distributor of electricity to eastern parts of Sydney, the Central Coast, Newcastle and the Hunter Region.

In June 2014, the then government announced its commitment to lease components of the state's electricity network as part of the Rebuilding NSW plan. Implementation of the policy began after the government was re-elected in 2015. Between November 2015 and August 2016, the NSW Government held a competitive tender process to lease 50.4 per cent of Ausgrid for 99 years. The NSW Government abandoned the process on 19 August 2016 after the Australian Treasurer rejected two bids from foreign investors, for national security reasons. That day, the Premier and Treasurer released a media statement clarifying the government's objective to complete the transaction via a competitive process in time to include the proceeds in the 2017–18 budget.

On 31 August 2016, the state received an unsolicited proposal from IFM Investors and AustralianSuper to acquire an interest in Ausgrid under the same terms proposed by the state during the tender process. In October 2016, the government accepted the unsolicited proposal. 

This audit examined whether the unsolicited proposal process for the partial long-term lease of Ausgrid was effectively conducted and in compliance with the government’s 2014 Unsolicited Proposals: Guide for Submission and Assessment (Unsolicited Proposals Guide or the Guide). 

The audit focused on how the government-appointed Assessment Panel and Proposal Specific Steering Committee assessed key requirements in the Guide that unsolicited proposals must be demonstrably unique and represent value for money. 

Appendix one - Response from agency

Appendix two - NSW Government’s summary of assessment of the Ausgrid unsolicited proposal   

Appendix three - The definition and nature of unsolicited proposals

Appendix four - Ausgrid unsolicited proposal process

Appendix five - About the audit

Appendix six - Performance auditing

Parliamentary reference - Report number #309 - released 11 December 2018

The Premier’s Implementation Unit uses a systematic approach to measuring and reporting progress towards the Premier’s Priorities performance targets, but public reporting needed to improve, according to a report released today by the Auditor-General of NSW, Margaret Crawford.

The Premier of New South Wales has established 12 Premier’s Priorities. These are key performance targets for government.

Source: Department of Premier and Cabinet, Premier’s Priorities website.

Each Premier’s Priority has a lead agency and minister responsible for achieving the performance target.

The Premier’s Implementation Unit (PIU) was established within the Department of Premier and Cabinet (DPC) in 2015. The PIU is a delivery unit that supports agencies to measure and monitor performance, make progress toward the Premier’s Priorities targets, and report progress to the Premier, key ministers and the public.

This audit assessed how effectively the NSW Government is progressing and reporting on the Premier's Priorities.

The Premier’s Implementation Unit (PIU) is effective in assisting agencies to make progress against the Premier’s Priorities targets. Progress reporting is regular but transparency to the public is weakened by the lack of information about specific measurement limitations and lack of clarity about the relationship of the targets to broader government objectives.The PIU promotes a systematic approach to measuring performance and reporting progress towards the Premier’s Priorities’ performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives.

The PIU provides a systematic approach to measuring performance and reporting progress towards the Premier's Priorities performance targets. Public reporting would be improved with additional information about the rationale for choosing specific targets to report on broader government objectives. The data used to measure the Premier’s Priorities comes from a variety of government and external datasets, some of which have known limitations. These limitations are not revealed in public reporting, and only some are revealed in progress reported to the Premier and ministers. This limits the transparency of reporting.

The PIU assists agencies to avoid unintended outcomes that can arise from prioritising particular performance measures over other areas of activity. The PIU has adopted a collaborative approach to assisting agencies to analyse performance using data, and helping them work across organisational silos to achieve the Premier’s Priorities targets.

Data used to measure progress for some of the Premier’s Priorities has limitations which are not made clear when progress is reported. This reduces transparency about the reported progress. Public reporting would also be improved with additional information about the relationship between specific performance measures and broader government objectives.

The PIU is responsible for reporting progress to the Premier, key ministers and the public. Agencies provide performance data and some play a role in preparing progress reports for the Premier and ministers. For 11 of the Premier's Priorities, progress is reported against measurable and time-related performance targets. For the infrastructure priority, progress is reported against project milestones.

Progress of some Priorities is measured using data that has known limitations, which should be noted wherever progress is reported. For example, the data used to report on housing completions does not take housing demolitions into account, and is therefore overstating the contribution of this performance measure to housing supply. This known limitation is not explained in progress reports or on the public website.

Data used to measure progress is sourced from a mix of government and external datasets. Updated progress data for most Premier’s Priorities is published on the Premier’s Priorities website annually, although reported to the Premier and key ministers more frequently. The PIU reviews the data and validates it through fieldwork with front line agencies. The PIU also assists agencies to avoid unintended outcomes that can arise from prioritising single performance measures. Most, but not all, agencies use additional indicators to check for misuse of data or perverse outcomes.

We examined the reporting processes and controls for five of the Premier’s Priorities. We found that there is insufficient assurance over the accuracy of the data on housing approvals.

The relationships between performance measures and broader government objectives is not always clearly explained on the Premier’s Priority website, which is the key source of public information about the Premier’s Priorities. For example, the Premier’s Priority to reduce litter volumes is communicated as “Keeping our Environment Clean.” While the website explains why reducing litter is important, it does not clearly explain why that particular target has been chosen to measure progress in keeping the environment clean.

By December 2018, the Department of Premier and Cabinet should:

1. improve transparency of public reporting by:
   • providing information about limitations of reported data and associated performance
   • clarifying the relationship between the Premier’s Priorities performance targets and broader government objectives.
2. ensure that processes to check and verify data are in place for all agency data sources
3. encourage agencies to develop and implement additional supporting indicators for all Premier’s Priority performance measures to prevent and detect unintended consequences or misuse of data.

The Premier's Implementation Unit is effective in supporting agencies to deliver progress towards the Premier’s Priority targets.

The PIU promotes a systematic approach to monitoring and reporting progress against a target, based on a methodology used in delivery units elsewhere in the world. The PIU undertakes internal self-evaluation, and commissions regular reviews of methodology implementation from the consultancy that owns the methodology and helped to establish the PIU. However, the unit lacks periodic independent reviews of their overall effectiveness. The PIU has adopted a collaborative approach and assists agencies to analyse performance using data, and work across organisational silos to achieve the Premier’s Priorities targets.

Agency representatives recognise the benefits of being responsible for a Premier's Priority and speak of the value of being held to account and having the attention of the Premier and senior ministers.

By June 2019, the Department of Premier and Cabinet should:

1. establish routine collection of feedback about PIU performance including:
   • independent assurance of PIU performance
   • opportunity for agencies to provide confidential feedback.

Appendix one: Response from agency

Appendix two: Data sources

Appendix three: About the audit

Appendix four: Performance auditing

Parliamentary reference - Report number #307 - released 13 September 2018

Infrastructure NSW effectively manages how grant applications for regional assistance programs are assessed and recommended for funding. Its contract management processes are also effective. However, we are unable to conclude whether the objectives of these programs have been achieved as the relevant agencies have not yet measured their benefits, according to a report released today by the Auditor-General for New South Wales, Margaret Crawford. 

In 2011, the NSW Government established Restart NSW to fund new infrastructure with the proceeds from the sale and lease of government assets. From 2011 to 2017, the NSW Government allocated $1.7 billion from the fund for infrastructure in regional areas, with an additional commitment of $1.3 billion to be allocated by 2021. The NSW Government allocates these funds through regional assistance programs such as Resources for Regions and Fixing Country Roads. NSW councils are the primary recipients of funding provided under these programs.

The NSW Government announced the Resources for Regions program in 2012 with the aim of addressing infrastructure constraints in mining affected communities. Infrastructure NSW administers the program, with support from the Department of Premier and Cabinet.

The NSW Government announced the Fixing Country Roads program in 2014 with the aim of building more efficient road freight networks. Transport for NSW and Infrastructure NSW jointly administer this program, which funds local councils to deliver projects that help connect local and regional roads to state highways and freight hubs.

This audit assessed whether these two programs (Resources for Regions and Fixing Country Roads) were being effectively managed and achieved their objectives. In making this assessment, we answered the following questions:

• How well are the relevant agencies managing the assessment and recommendation process?
• How do the relevant agencies ensure that funded projects are being delivered?
• Do the funded projects meet program and project objectives?

The audit focussed on four rounds of Resources for Regions funding between 2013–14 to 2015–16, as well as the first two rounds of Fixing Country Roads funding in 2014–15 and 2015–16.

Appendix one - Response from agencies

Appendix two - Maps of funded projects

Appendix three - About the audit

Appendix four - Performance auditing

Parliamentary reference - Report number #300 - released 17 May 2018

The 'Stronger Councils, Stronger Communities' and the 'Dogs deserve better' government advertising campaigns complied with the Government Advertising Act and most elements of the Government Advertising Guidelines.

However, some advertisements were designed to build support for government policy and used subjective or emotive messages. This is inconsistent with the requirement in the Government Advertising Guidelines for 'objective presentation in a fair and accessible manner'.

Advertisements in the 'Stronger Councils, Stronger Communities' campaign used subjective statements such as 'the system is broken' and 'brighter future'. While advertisements in the 'Dogs deserve better' campaign used confronting imagery such as gun targets, blood smears and gravestones.

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit in relation to at least one government advertising campaign in each financial year. The performance audit assesses whether advertising campaigns were carried out effectively, economically and efficiently and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). In this audit, we examined two campaigns:

• the ‘Stronger Councils, Stronger Communities’ campaign run by the Office of Local Government and the Department of Premier and Cabinet
• the ‘Dogs deserve better’ campaign run by the Department of Justice.    

Section 6 of the Act details the specific prohibitions on political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed so as to influence (directly or indirectly) support for a political party.

The ‘Stronger Councils, Stronger Communities’ government advertising campaign was run by the Office of Local Government and the Department of Premier and Cabinet in four phases from August 2015 to May 2016. The total cost of the campaign was over $4.5 million. See Appendix 2 for more details on this campaign.

No breach of specific prohibitions in the Act

Section 6 of the Act prohibits the use of government advertising for political advertising. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, any other member of parliament or a candidate nominated for election to parliament
• contain the name, logo or any slogan of, or any other reference relating to, a political party.

We did not identify any breach of the specific prohibitions listed above in the advertising content of this campaign.

Request for quotes to design advertisement did not reflect the full scope required

The request for quotes for the design of the Phase 1 advertisement did not reflect the full scale of work that was to be undertaken, and this created a risk to achieving value for money. The Office of Local Government sought quotes for design of a television advertisement only. It did not request an estimate for radio, online advertisements, or translation for linguistically diverse audiences, which were ultimately required for the campaign.
 

A full and fair assessment of which supplier could provide the best value for money could not be made given that the quotes obtained did not reflect the full scope of work. The final amount paid for the design of Phase 1 was 2.7 times the original quote. It is possible that another supplier that provided a quote could have provided overall better value for money.

The Office of Local Government continued to use the Phase 1 supplier for Phase 2 and Phase 2 extension (Exhibit 4). Where there are other suppliers that could feasibly compete for a contract, direct negotiation increases the risk the agency has not obtained the best value for money. The department advised that it continued with the same agency to avoid costs involved in briefing a new agency on the campaign.

The ‘Dogs deserve better’ government advertising campaign was run by the Department of Justice from August 2016, after the government announced its decision to prohibit greyhound racing, and was terminated in October 2016 after a change of government policy. The campaign had a budget of $1.6 million, with an actual spend of $1.3 million. See Appendix 2 for more details on this campaign.

No breach of specific prohibitions in the Act

Section 6 of the Act prohibits the use of government advertising for political advertising. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, any other member of parliament or a candidate nominated for election to parliament
• contain the name, logo or any slogan of, or any other reference relating to, a political party.

We did not identify any breach of the specific prohibitions listed above in the advertising content of this campaign.
 

Animal welfare concerns were identified as the reason for urgent advertising

A brief prepared by the department in July 2016 raised concerns about the welfare of greyhounds following the NSW Premier’s announcement that the government would prohibit greyhound racing. The brief raised the risk that industry members may make impulse decisions to destroy their greyhounds without information on support that was being offered.

The department used the provisions in Sections 7(4) and 8(3) of the Act to expedite the release of advertising due to ‘other urgent circumstances’. This provision allows advertising to commence prior to completing the peer review process and cost benefit analysis.
 
In introducing the Government Advertising Bill to parliament in 2011, the then Premier noted that exceptional circumstances would cover situations ‘such as a civil emergency or sudden health epidemic’. There is no other guidance on when it is appropriate to use this section. It is at the discretion of a government agency head to determine whether a campaign is urgent.
 

Phase 1 advertisements did not focus on the urgent needs

This advertising campaign had three overarching objectives:

• to increase public awareness of the animal welfare reasons for the closure of the greyhound racing industry
• to change the behaviour of dog owners from potentially harming their greyhounds to treating them humanely, by accessing the support options and packages available
• to promote greyhound adoptions by the public.

Alongside advertising, the department took other steps to engage with the greyhound racing industry. This included direct mail, face to face meetings around the State, setting up a call centre and community consultation through an online survey. Other government agencies and animal welfare agencies were also engaged to reach out to affected stakeholders.

Phase 1 advertising content focused on providing information about the reasons for the closure of the industry. The department’s radio and television advertisements did not refer to support packages or encourage the public to adopt a greyhound. While print advertisements did mention these things, this was only presented in fine print. In all advertisements, audiences were referred to a website for further information.

The focus of advertisements on the reasons for industry closure was not consistent with the identified needs to urgently commence advertising to influence the behaviour of dog owners and encourage the public to adopt a greyhound.

The content in Phase 2 advertisements, which began around four weeks after the first phase, was more explicit in highlighting the services and support for industry members such as offering business and retraining advice. These advertisements also referred audiences to a call centre number as well as the website.

Peer review process limited to influencing second phase of advertisements

In urgent circumstances, the Act allows for peer review to be completed after advertising has commenced. For this campaign, the peer review process was completed on 19 August 2016, two weeks after advertising had commenced. Where advertising commences before the peer review process is completed, the usefulness of peer reviewers’ recommendations is limited to informing subsequent phases of advertising and the post-campaign evaluation.

The peer review report found the messages in Phase 1 advertisements were not clearly defined, and the role of advertising was not clearly defined amongst other campaign activities. These recommendations informed the second phase of advertising, which ran from 27 August 2016 until the campaign was terminated in October 2016.
 

The department could not demonstrate value for money was achieved for creative work

The department provided a fixed budget for creative work when requesting quotes from creative agencies to develop advertising material. This is not consistent with the quotation requirements in the government’s Guidelines for Advertising and Digital Communication Services. This approach creates risks to achieving value for money as creative agencies are not required to compete on price for their services. The department advised that it had pre-set the creative costs based on a comparative government campaign of a similar size. This was done due to a concern that requiring agencies to compete on price would affect the short timeframe given to develop creative material.

Three creative agencies accepted the opportunity to present design ideas for the campaign. The department was unable to provide evidence of how it chose the preferred supplier out of these three agencies. Records are important for accountability and allow a procurement decision to be audited after an urgent decision.     
 

Short notice did not allow for cost-efficient media booking for all phases

Placement of advertisements in various media channels was done through the State’s Media Agency Services contract. This contract achieves savings as the government can use its aggregated media spend to gain discounts from the media supplier.

The Department of Premier and Cabinet provides guidance to ensure cost efficient media booking. For example, media time for a television advertisement should be booked at least 6 to 12 weeks in advance. Radio advertisements should be booked at least 2 to 8 weeks in advance.

The peer review report noted that the department did not have adequate time to look for the most cost-efficient way to advertise. In its response to the peer reviewers, the department acknowledged this to be due to the urgency to start advertising. The media booking authority was signed by the department one day before the campaign commenced.
 

The department used a wide public campaign for a narrow target audience

The campaign identified greyhound industry participants as the primary target audience. In 201516 there were 1,342 greyhound trainers, 1,695 owner/trainers, 983 attendants and 1,247 breeders in New South Wales. The department’s advertising submission identified ‘concerns that industry members could make impulsive decisions, potentially jeopardising the welfare of a large number of dogs, prior to the shutdown of the industry’.

The submission’s evidence of advertising effectiveness focused on increasing the level of wider community support for the ban rather than stopping industry members from making impulse decisions. It used an early opinion poll to show that total support for the ban on greyhound racing rises by 17 points and opposition drops by four points following explanation of the findings of the Special Commission of Inquiry report.

The peer review report noted that the role of advertising was not clearly defined amongst the department’s range of other direct and targeted communications and consultations held with industry members.

No demonstrated basis for use of confronting imagery and provocative language

The Guidelines require ‘objective presentation in a fair and accessible manner’. Neither the Guidelines or Handbook further explain what objective presentation means. We have used an ordinary definition of this term as ‘not influenced by personal feelings or opinions in considering and representing facts’. This is synonymous with terms like ‘impartial’, ‘neutral’, and ‘dispassionate’ and opposite to ‘subjective’. We consider that to meet the current requirements in the Guidelines for objectivity, advertising content should contain accurate statements or facts, and avoid subjective language.

Phase 1 focussed on the ongoing consequences if no action was taken to close the industry. The advertisements used provocative language, for example ‘Up to 70 per cent of dogs are deemed wastage by their own industry. Wastage! Slaughtered just for being slow’. Advertisements used confronting imagery like gravestones, blood smears and gun targets.

Our literature review into this area highlighted mixed findings on the effectiveness of confrontational advertising materials. In some cases, shock campaigns may cause an audience to reject or ignore the message, and may even encourage people to do the opposite of the intended behaviour. In other cases, such as in road safety campaigns, this style of advertising can be successful. This shows the importance of conducting pre-campaign research before adopting a confrontational or emotive approach in advertising.

The Government Advertising Handbook recommends that an agency explain the rationale and the evidence for their chosen advertising approach. There was no evidence that the department researched the effectiveness of its advertising approach with its target audience. The department had planned to undertake creative concept testing as part of a strategy to ensure the creative material was understood by its audience. The department advised that due to the urgency of the campaign, it did not have time to conduct this testing.

Not all Phase 1 radio advertisements clearly identified that they were authorised by the New South Wales Government

For the first few days on air, Phase 1 radio advertisements ended by referring the audience to a government website, instead of clearly identifying that it had been authorised by the New South Wales Government. Government authorisations and logos ensure the work and the programs of the NSW Government are easily identifiable by the community.    

The department’s cost benefit analysis did not consider alternatives to advertising

For government advertising campaigns that cost over $1.0 million, the Act requires the advertising agency to carry out a cost benefit analysis and obtain approval from the Cabinet Standing Committee on Communications, prior to commencing the campaign.

The department engaged with audiences through direct mail, face to face forums, and a telephone helpline in addition to advertising. However, the department’s cost benefit analysis did not meet the requirements in the Guidelines to specify the extent to which expected benefits could be achieved without advertising, and to compare costs of options other than advertising that could be used to successfully implement the program (see Exhibit 6).

The cost benefit analysis made optimistic assumptions about the impact of the campaign on greyhound adoptions. It estimated that 2,360 greyhounds would be adopted if the campaign was run. This is significantly higher than the ‘most optimistic outcome’ of re-homing in the Special Commission Inquiry report (we calculated this to be 1,467 greyhounds). There was insufficient evidence to support the higher number of adoptions in the cost benefit analysis.

The sensitivity analysis shows that using the Special Commission’s ‘most optimistic outcome’ figure of re-homing would reduce the net present value of advertising to be negative. Further, the cost benefit analysis also assumed that increased government funding would be made available to animal welfare and rehoming organisations to support more adoptions, but did not estimate or include this cost when calculating the net present value of advertising.
 

There were two factual inaccuracies in key messages used for Phase 1 advertisements

Section 8(2) of the Act requires the head of a government agency to certify that the proposed campaign ‘contains accurate information’. The Secretary of the Department of Justice signed the compliance certificate on 29 July 2016, before advertisements commenced.

We examined the accuracy of factual claims in this advertising campaign, by comparing the key statements to the report of Special Commission of Inquiry into the Greyhound Racing Industry (the Commissioner report). The Commissioner report was quoted by the NSW Government as the basis for its policy to transition the greyhound racing industry to closure.

We identified that two of the key statements used in Phase 1 advertisements to support the animal welfare reasons for industry closure were inaccurate (Exhibit 7).    

Appendix one - Responses from agencies

Appendix two - About the campaigns

Appendix three - About the Audit

Appendix four - Performance Auditing

Parliamentary reference - Report number #294 - released 2 November 2017

The processes used to assess NorthConnex adequately considered value for money for taxpayers.This report also found that the impact of tolling concessions on road users and the motorway network was consistent with policy objectives described in the 2012 NSW Long Term Transport Master Plan.

Parliamentary reference - Report number #287 - released 8 June 2017