Media release
A report released today by the Auditor-General for New South Wales, Margaret Crawford found that select advertising campaigns conducted by Service NSW and the NSW Rural Fire Service met most requirements of the Government Advertising Act, regulations, Guidelines and other laws. However, the audit found that Service NSW inappropriately used its post campaign evaluation to measure sentiment towards and confidence in the NSW Government.
While agency analysis shows that the ‘Cost of Living’ (phases 2 and 3) and ‘How Fireproof is Your Plan?’ campaigns achieved most of their objectives, the campaign objectives and targets set by both agencies were not sufficient to measure all aspects of campaign effectiveness.
The report makes two recommendations to the Department of Customer Service. The first is to review its guidance to ensure agencies are not using post campaign evaluations to measure sentiment towards the government. The second, to review its guidance and the new process of peer review to ensure they support agencies to comply with the Act, the regulations and the Guidelines.
The Government Advertising Act 2011 requires the Auditor General to conduct an annual performance audit of one or more government agencies to see whether their advertising activities were carried out in an effective, economical and efficient manner and in compliance with the Government Advertising Act 2011.
Executive Summary
The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). This audit examined two campaigns run during the 2018–19 and 2019–20 financial years respectively:
- the 'Cost of Living' campaign run by Service NSW (phases 2 and 3 delivered in 2018–19)
- the 'How Fireproof Is Your Plan?' (Fireproof) campaign run by NSW Rural Fire Service (year two of a three-year campaign delivered in 2019–20).
Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.
Conclusion
Neither campaign breached the prohibition on political advertising contained in section 6 of the Act. While both campaigns met most requirements of the Act, the regulations, other laws and the Guidelines, we identified some instances of non-compliance. Service NSW inappropriately used its post campaign evaluation to measure sentiment towards and confidence in the NSW Government.
Service NSW used its post-campaign evaluation to measure sentiment towards and confidence in the NSW Government. While neither campaign breached the prohibition on political advertising contained in section 6 of the Act, measuring sentiment towards and confidence in the NSW Government is not an appropriate use of the post-campaign evaluation and creates a risk that the results may be used for party political purposes. This risk is heightened as both phases 2 and 3 of the Cost of Living campaign were run immediately before the NSW state election. We have made this finding previously in our report 'Government advertising 2017–18'.
The campaign objectives and targets set by both agencies were not sufficient to fully measure campaign effectiveness. Service NSW advertised seven rebates in phase 2 of the campaign but only set targets for the awareness and uptake of three of these rebates. NSW Rural Fire Service set objectives and targets to be achieved over the life of the three-year campaign but did not set targets to be achieved for each year of the campaign. While the Fireproof campaign is a three-year campaign, each year of the campaign is subject to a separate approval and peer review process.
Agency analysis shows that both campaigns achieved most of their objectives. There was some overlap in the timing of phases 2 and 3 of the Cost of Living campaign and both phases had similar high-level objectives to increase awareness of rebates, making it difficult to evaluate the effectiveness of each distinct campaign phase. NSW Rural Fire Service conducted a post-campaign evaluation for year two of the Fireproof campaign (2019–20) but although this showed positive results against the overall objectives of the three-year campaign, NSW Rural Fire Service did not set specific targets for year two of the campaign, making it difficult to evaluate effectiveness for that year.
Service NSW was not able to demonstrate that its campaign was economical as it directly negotiated with a single supplier for the creative materials for phase 2. This is contrary to the NSW Government's procurement rules which require agencies to obtain three quotes when using suppliers on a prequalification scheme. Service NSW did not comply with its own procurement policy, which restricts Service NSW employees from entering into discussions with a supplier until the appropriate delegate approves a direct procurement. NSW Rural Fire Service achieved cost efficiencies by re-using creative material developed in the first year of the campaign. NSW Rural Fire Service also received $4 million worth of free advertising time and space.
The cost benefit analyses prepared by both agencies did not fully meet the requirements in the Guidelines. Both agencies identified an alternative to advertising but did not assess the costs and benefits of that alternative. We have made this finding previously in our report 'Government advertising 2017–18' and in our report 'Government advertising 2015–16 and 2016–17'.
1. Recommendations
By 30 June 2021, the Department of Customer Service should:
- update its guidance to ensure that post-campaign evaluations are not used to measure sentiment towards the government
- review its guidance and the new process for campaign review to ensure that they support agencies to comply with all requirements set out in the Act, the regulations and the guidelines including common areas of deficiency such as:
- the development of campaign objectives and targets that are sufficient to fully measure campaign effectiveness
- identifying options other than advertising that could be used to achieve campaign objectives and assessing the costs and benefits of these options when a cost benefit analysis is required under the Act.
1. Introduction
1.1 Background
What is government advertising?
Governments use advertising to communicate information about a government program, policy or initiative to members of the public. Government advertising is funded by or on behalf of a government agency and can be distributed through a variety of media, such as radio, television, the Internet, newspapers, billboards or cinemas.
There are three broad categories of government advertising:
- recruitment advertising - advertising which promotes specific job vacancies and employment opportunities within a government agency
- public notices - advertising which communicates a clear, simple message or announcement and is generally one-off or short-term in nature
- public awareness advertising - co-ordinated communications to raise awareness of key issues, such as government initiatives, or social marketing to encourage behaviour change.
How much is spent on government advertising in New South Wales?
The NSW Government spent $83.8 million on advertising in 2018–19 and $69.8 million on advertising in 2019–20.
How is government advertising regulated?
A regulatory framework which includes both policy and legislation governs NSW Government advertising. Exhibit 2 contains an overview of this regulatory framework.
Regulation | Purpose |
---|---|
Government Advertising Act 2011 (Act) | Sets out the legal requirements for government advertising |
Government Advertising Regulation 2018 (Regulation)* | Sets out exemptions to the Act. |
NSW Government Advertising Guidelines 2012 (Guidelines) | Sets out requirements in relation to the style and content, dissemination and cost of government advertising campaigns, as well as the requirements of cost benefit analyses and peer reviews. |
Note: The Government Advertising Handbook no longer applies. The NSW Government website on government communications explains the legal and policy requirements of government advertising.
Source: Audit Office analysis.
Prohibition of political advertising
Section 6 of the Act prohibits political advertising as part of a government advertising campaign. Government advertising campaigns must not:
- be designed to influence (directly or indirectly) support for a political party
- contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
- contain the name, logo, slogan or any other reference to a political party.
In addition, the Guidelines require government advertising campaigns to be politically neutral and clearly distinguishable from party political messages.
Requirements prior to the commencement of a campaign
The Act states that a government advertising campaign must not commence unless the head of the agency has signed a compliance certificate for the campaign. This compliance certificate states that the head of the agency believes the government advertising campaign:
- complies with the Act, Regulation and Guidelines
- contains accurate information
- is necessary to achieve a public purpose and is supported by analysis and research
- is an efficient and cost-effective means of achieving its public purpose.
The Act defines further requirements for campaigns which are likely to exceed a total cost of $50,000 or $1.0 million. The Government Advertising Regulation 2018 revised these thresholds to $250,000 and $1.0 million which apply for campaigns approved after 21 December 2018. These are summarised in Exhibit 3.
Government advertising campaigns likely to cost over $250,000 (or $50,000 if approved prior to 21 December 2018) are subject to peer review before the campaign commences. The peer review process changed in 2019–20.
In 2018–19, the Department of Premier and Cabinet managed the peer review process. For campaigns approved in 2018–19, the peer review process involved two or three public sector employees with expertise in marketing and communications assessing the proposed advertising campaign according to a set of criteria outlined in the Guidelines.
From 2019–20, the Department of Customer Service manages the peer review process. This was a result of machinery of government (MoG) changes which moved the functional group managing the peer review process from the Department of Premier and Cabinet to the Department of Customer Service. Peer review is a two-stage process:
- Budget approval - government agencies submit an advertising budget proposal to the Department of Customer Service Campaign Effectiveness Team. The Campaign Effectiveness Team review the budget proposal and make recommendations to the Delivery and Performance Committee of Cabinet (DaPCo). DaPCo is responsible for approving campaign budgets.
- Campaign review - after DaPCo approves a campaign budget, the Department of Customer Service Effectiveness Team conduct a review of the proposed campaign. The Campaign Effectiveness Team assess the need for the proposed advertising campaign, the creative and media strategy (including objectives and target audiences) and how the agency will manage the campaign.
The Act requires an agency wishing to run a government advertising campaign that is likely to cost over $1.0 million to complete a cost-benefit analysis before the campaign commences. A cost-benefit analysis aims to evaluate the net economic cost or benefit of the campaign and indicate to decision makers how the campaign will affect the wellbeing of NSW residents.
Advertising cost | Compliance certificate | Budget approval by DaPCo** | Campaign review | Cost-benefit analysis |
---|---|---|---|---|
<$250,000 | ||||
$250,000* to $1.0 million | ||||
>$1 million |
** The budget approval process was introduced for campaigns approved in 2019–20. Prior to 2019–20, campaign budgets over $1,000,000 were approved by the Cabinet Standing Committee on Communications and Government Advertising.
Source: Audit Office analysis.
About this audit
The Act requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year.
In conducting the audit, the Auditor-General must determine whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the Guidelines and other laws.
The 2018–19 and 2019–20 government advertising campaigns examined in this audit are listed in Exhibit 4.
Year | Campaign title | Responsible agency | Expenditure |
---|---|---|---|
2018–19 | Cost of Living (phases 2 and 3) | Service NSW | $4,030,769 (phase 2) $926,032 (phase 3) |
2019–20 | How FireProof Is Your Plan? | NSW Rural Fire Service | $1,401,750.06 |
Appendix two contains details about each campaign.
Appendix three contains further details about this audit.
2. Cost of Living campaign
In 2018–19, Service NSW delivered phases 2 and 3 of the 'Cost of Living' campaign. The Cost of Living advertising campaign aimed to build awareness of the help available to ease the cost of living for people under financial pressure including awareness of specific rebates that can be claimed. As part of the Cost of Living program, Service NSW developed a webpage designed as a single portal to access more than 40 NSW Government savings, rebates and initiatives (which originated from over 12 different agencies). It also launched the Cost of Living service which includes face to face meetings and phone interviews to help people claim rebates from the NSW Government. Phase 2 of the campaign ran from September 2018 to August 2019. Phase 3 of the campaign ran from January 2019 to July 2019. The budgets for phases 2 and 3 were $4.127 million and $934,800 respectively. See Appendix two for more details on this campaign.
Campaign materials we reviewed did not breach section 6 of the Act
The audit team reviewed campaign materials developed as part of the paid advertising campaign including radio transcripts, digital videos and display. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. See Appendix two for examples of campaign materials for this campaign.
Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:
- be designed to influence (directly or indirectly) support for a political party
- contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
- contain the name, logo, slogan or any other reference to a political party.
The audit found no breaches of section 6 of the Act in the campaign material we reviewed.
Post-campaign evaluations measured sentiment towards and confidence in the NSW Government
The post-campaign evaluation for phases 2 and 3 measured levels of confidence with the statement ‘the NSW Government has your best interests at heart’, despite the fact this was not a stated objective of the campaign. This is not an appropriate use of the post-campaign evaluation, which should measure the success of the campaign against its stated objectives. The post-campaign evaluation for phase 3 found that exposure to the campaign improved sentiment towards the government amongst those who did not have confidence in the NSW Government.
Service NSW advised that it was important to measure the sentiment of the advertising including the wording 'best interests' as it did not want the whole of government brand to be detrimental to customer engagement with applying for the rebates.
Following phase 2, Service NSW conducted analysis of media sentiment using the key words 'cost of living' and the names of the Premier, Treasurer and Minister for Customer Service. The analysis presented the level of positive, negative and neutral media sentiment. The Government Advertising Guidelines 2012 list the purposes that government advertising may serve which do not include improving the perception of the government. The inclusion of this analysis in Service NSW's post-campaign evaluation creates a risk that the results may be used for party political purposes.
Section 10 of the Act restricts agencies from carrying out a campaign after 26 January in the calendar year before the Legislative Assembly is due to expire and before the election for the Legislative Assembly in that year. Service NSW authorised a media agency to book media in line with the media plans for the campaign. The media plans for the campaign show that Service NSW did not authorise or plan to run any advertisements between 27 January 2019 and 23 March 2019.
Service NSW did not set targets for all rebates advertised in phase 2
Service NSW did not set targets for four of the seven rebates that were advertised as part of phase 2 of the campaign. These rebates were the Family Energy Rebate, Appliance Replacement Offer, National Parks Concession Offer and the Pensioner Travel Voucher. As a result, it was unable to evaluate whether the advertisements for these rebates were effective. Service NSW advised that at the time the campaign went to peer review, when campaign objectives are set, it did not know which rebates would be included in the advertisements.
Service NSW stated in its submission to the Department of Premier and Cabinet that it may change the creative content for phase 2 as it announced new initiatives and rebates. The peer review process should have ensured that Service NSW set targets for any additional rebates or savings it intended to advertise before that advertising commenced to ensure a strategic approach to the campaigns that clearly demonstrated anticipated benefits were in place.
The post-campaign evaluation for phase 2 shows that the advertising campaign met most of its objectives
Service NSW set overall campaign objectives and specific targets for some rebates advertised as part of phase 2 of the campaign. The objectives, targets and results for phase 2 are shown in Exhibit 5. In phase 2, Service NSW established baseline data on levels of awareness of government rebates during the peer review process. The baseline level of awareness for government rebates was 44 per cent. The level of awareness for specific rebates was 46 per cent for the Compulsory Third Party (CTP) green slip refund, and 21 per cent for both Active Kids and Toll Relief.
Post-campaign evaluation reports for phase 2 show that the campaign met its objective to raise awareness of NSW Government rebates, achieving a 16 per cent increase in awareness from 44 per cent to 51 per cent. The campaign did not meet its target to increase awareness of the CTP green slip refund by ten per cent.
Service NSW did not report the results of the uptake of the CTP green slip refund, Active Kids and Toll Relief in its post campaign effectiveness report submitted to the Department of Premier and Cabinet. However, other post-campaign evaluation documentation, which Service NSW advise was submitted to the Department of Premier and Cabinet, show that these targets were met.
Service NSW did not report to the Department of Premier and Cabinet on whether it achieved the target of a ten per cent increase of average monthly visits to the Cost of Living webpage. Service NSW reported that it had achieved an average of 11,753 visitors to the webpage per day during the campaign. These average daily results indicate that the target was met.
Campaign objectives and targets | Does the post-campaign evaluation show that the target was met? |
---|---|
1. a) Increase awareness of rebates from the NSW Government by ten per cent. | |
b) Increase average monthly visits to the Cost of Living webpage by ten per cent. |
|
2. Increase awareness of rebates and savings by ten per cent for: |
|
|
|
|
|
|
|
3. Increase awareness that NSW Government initiatives relating to the cost of living are available via Service NSW by ten per cent. | |
4. Increase the uptake of rebates and savings for the CTP green slip refund, Active Kids and Toll Relief by ten per cent. |
Key | Yes | Not Fully |
Source: Service NSW. Audit Office analysis.
The post-campaign evaluation for phase 3 shows that the advertising campaign met most of its objectives
Service NSW set overall campaign objectives and specific targets for the two rebates advertised as part of phase 3 of the campaign. The objectives, targets and results for phase 3 are shown in Exhibit 6.
In phase 3, Service NSW established baseline data on levels of awareness during the peer review process. The baseline level of awareness for government rebates was 44 per cent. This is the same baseline that was used to measure performance for phase 2 of the campaign. Service NSW did not set baselines for awareness and uptake of Energy Switch and Creative Kids as these were new services.
Post-campaign evaluation reports for phase 3 show that the campaign met its objective to raise awareness of NSW Government rebates by ten per cent, achieving a 30 per cent increase in awareness from 44 per cent to 57 per cent. The overall increase in message take-out was met with 43 per cent agreeing with the message that the NSW Government is taking steps to ease the cost of living. The campaign achieved awareness and uptake targets for the specific rebates included in phase 3, except for awareness of Creative Kids which achieved 28 per cent awareness, falling short of the 30 per cent awareness target.
Campaign objectives and targets | Does the post-campaign evaluation show that the target was met? |
---|---|
1. Increase message takeout that ‘The NSW Government is taking steps to help ease the cost of living in NSW’ by ten per cent for those who can recall the campaign. | |
2. Increase awareness that the NSW Government has a range of rebates and savings by ten per cent. | |
3. Generate awareness with NSW residents aged 18+ of: |
|
|
|
|
|
4. Create uptake of Energy Switch and Creative Kids (8,356 clicks on the Energy Switch website and 107,938 Creative Kids vouchers downloaded with 70 per cent conversion). |
Key | Yes | Not Fully |
The timing of campaign phases meant that it was difficult for Service NSW to evaluate each distinct campaign phase and reduced opportunities to incorporate learnings from previous phases
Service NSW commenced planning for phase 2 of the campaign while phase 1 was still underway. This limited the opportunity for Service NSW to incorporate learnings from phase 1 into phase 2. There was some overlap in the timing of phase 2 and the start of phase 3 of the campaign, making it difficult to evaluate the effectiveness of each distinct campaign phase. Both phases 2 and 3 had the same high-level outcome objective to raise awareness of rebates by ten per cent. The baseline measures that were used to evaluate performance for phase 3 were the same as those used to evaluate phase 2. As a result, Service NSW was not able to separately evaluate these two phases of the campaign. This is important given the budgets for phases 2 and 3 were $4.127million and $934,800 respectively.
Service NSW allocated 7.5 per cent of its media budget to communications with culturally and linguistically diverse (CALD) and Aboriginal audiences
The NSW Government CALD and Aboriginal Advertising Policy requires that agencies spend at least 7.5 per cent of an advertising campaign media budget on direct communications with CALD and Aboriginal audiences. Service NSW authorised a media company to book media in line with the media plans for the campaign. The media plans for phases 2 and 3 of the campaign indicate that Service NSW met this requirement, with 7.5 per cent of the budget allocated to these audiences in phase 2 and 10.4 per cent in phase 3.
The post campaign evaluation for phases 1 and 2 of the Cost of Living campaign contained a recommendation to look at other opportunities to reach CALD audiences. Effective communication with CALD audiences was particularly important in phase 3 of the campaign, where they made up 30 per cent of the target audience for the Creative Kids advertisement. The post-campaign analysis for phase 3 showed that the campaign performed well with some, but not all CALD audiences. The post-campaign analysis also showed low awareness and uptake with Aboriginal audiences. Pre-campaign focus groups in phase 3 found Aboriginal audiences had a negative reaction to the campaign tag line ‘NSW Government is helping with the cost of living’ however this tagline was still used in some advertisements in phase 3.
The cost-benefit analysis (CBA) for phase 2 did not accurately assess the benefits of the campaign and did not assess the costs and benefits of alternatives to advertising
Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. The CBA conducted by Service NSW for phase 2 includes $8 million in benefits attributed to the advertisements for the Energy Switch tool and $6.9 million in benefits attributed to the advertisements for Creative Kids vouchers. These benefits should not have been included in the CBA for phase 2 as they were not included in this phase of the campaign. The CBA did not estimate the benefits of some other rebates and savings advertised in phase 2 of the campaign. This means that the CBA did not accurately assess the benefits of the campaign. Service NSW advised that at the time the CBA was developed it had not selected the rebates to be included in the campaign.
The Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits. The CBA developed as part of phase 2 identified using existing NSW Government communication channels as an alternative to advertising but did not assess the costs and benefits of this alternative.
This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.
Service NSW negotiated with a single creative agency in phase 2, making it difficult to demonstrate value for money
Agencies are required to obtain three quotes when procuring a creative agency on the prequalification scheme if the estimated cost of the creative content is greater than $150,000. In phase 2 of the campaign, Service NSW extended the contract with the creative agency used for phase 1 of the campaign and did not obtain three quotes despite the cost of the creative content for phase 2 being $731,480. The requirement to obtain three quotes was met in phase 1 when initially selecting this creative agency.
Service NSWs procurement policy details that direct negotiation may be appropriate where there is a compelling reason to renew or rollover a contract beyond temporal or convenience reasons or in the cases of a genuine emergency. In its briefing to the Chief Executive, Service NSW stated that this contract extension was sought due to the time-sensitive nature of the project and that if work was delayed by a tender process, Service NSW may not be able to meet marketing milestones and this could result in limited customer uptake. This reason is not a genuine emergency and is not compelling as it does not explain what consequences would occur if it did not meet the marketing milestones or if there was limited customer uptake.
Service NSW's procurement policy also states that under no circumstances must Service NSW employees enter into discussions with a supplier until the delegate has formally made their decision to enter into direct negotiation. Service NSW briefed the Chief Executive of Service NSW in relation to extending the contract on 5 September 2018. The briefing states that the creative agency had already begun developing creative content for phase 2 and Service NSW had already received quotes from the creative provider for the proposed work prior to 5 September 2018. Procurement sign-offs were not completed until 7 September 2018. The engagement of the creative provider prior to appropriate approvals was contrary to Service NSWs procurement policy.
The economy of the campaign may have been limited by not meeting the procurement requirements in phase 2. It is possible that the creative provider may have offered a more competitive rate if it was aware that Service NSW was seeking quotes from other creative providers. Additionally, it is possible that another creative provider could have provided better value for money.
In phase 3 of the campaign, the estimated cost of the creative exceeded $150,000 however Service NSW chose to contract two different creative agencies, and the cost for each agency fell below the threshold to obtain three quotes. Agencies are permitted to obtain one quote when using a creative provider on the prequalification scheme if the cost is between $50,000 to $150,000. Service NSW advised that it contracted two creative providers as two different project teams were responsible for the rebates, each with separate marketing budgets.
Service NSW allowed sufficient time for cost-efficient media placement
During the peer review process, the Department of Premier and Cabinet advised agencies about the time they should allow to ensure cost-efficient media placement. For example, the Department of Premier and Cabinet advised that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance.
Service NSW allowed sufficient time between the completion of the peer review process and the commencement of the first advertising. Service NSW signed the agreement with the approved Media Agency Services provider with sufficient time to achieve cost-efficient media placement for all types of media used in this campaign.
The campaign may have been misleading for some people who were not eligible for rebates
Advertisements we reviewed focused on the amount of savings that could be obtained from rebates, for example ‘Save up to $285’, and ended with a statement ‘To save, visit service.nsw.gov.au. This directed viewers to the Cost of Living website which contains eligibility information. However, the advertisements in phases 2 and 3 we reviewed did not contain any details on the eligibility for these rebates and not all advertisements stated that eligibility criteria apply. Service NSW advised that the eligibility criteria for each rebate is extensive and that it was not possible to include this in the creative material.
Post-campaign evaluations in phase 3 recommended that advertisements for Creative Kids should indicate eligibility (e.g. age criteria) as statements on savings have the potential to be misleading when not all viewers will be eligible for rebates. Social media analysis conducted following phase 2 showed ineligibility or inability to claim rebates or refunds caused anger for some respondents.
Some advertisements in phase 2 stated ‘we've got something for everyone’. However, as rebates were subject to eligibility criteria, it is possible that some residents in NSW would not be eligible for any rebates as part of the Cost of Living initiative. As such, this statement has the potential to be misleading.
The campaign included statements that underestimated the savings that some customers could obtain
The Guidelines require accuracy in the presentation of all facts, statistics, comparisons and other arguments. The Guidelines also require that all claims of fact included in government advertising campaigns must be able to be substantiated.
In phase 2, the possible savings customers could obtain for two rebates or savings exceeded the amounts stated in the advertising campaign. Exhibit 7 shows some advertisements in phase 2 which stated, ‘My Green Slip Saving Save up to $60’. However, the State Insurance Regulatory Authority website shows that savings for some types of motor vehicles under the 2017 CTP scheme exceed $60. The State Insurance Regulatory Authority website states that the average saving under this scheme has been $129. Service NSW advised that these advertisements were designed for regional markets and that it used different advertisements for metropolitan areas which contained different amounts of savings.
Some advertisements in phase 2 stated, ‘My Toll Relief save up to $700’. The Service NSW website states that drivers can obtain free vehicle registration if they have spent $1,352 or more in tolls in the previous financial year. The cost of registration for some vehicles exceeds $700. This means the savings detailed in the advertisement were lower than what some customers could actually save.
The cluster secretary signed the compliance certificate instead of the head of Service NSW
Under the Act, the head of an agency must sign a compliance certificate prior to commencement of the campaign. In phase 2 of the campaign, the Secretary of the former Department of Finance, Services and Innovation signed the compliance certificate. At the time the campaign was approved, Service NSW was part of the Finance, Services and Innovation cluster. The Public Sector Employment and Management (Service NSW Division) Order 2013 established Service NSW as an executive agency of the Division of the Government Service responsible to the Premier. Given this, the Chief Executive of Service NSW was responsible for signing the compliance certificate for phase 2.
Service NSW advised that at the time the campaign was approved, the Chief Executive of Service NSW reported to the Secretary of the former Department of Finance, Services and Innovation and that the former Department of Finance, Services and Innovation was the parent budget holder of the campaign. This is a minor non-compliance with the Act.
3. How FireProof Is Your Plan? campaign
NSW Rural Fire Service conducted the 'How FireProof Is Your Plan?' (Fireproof) campaign. The Fireproof campaign is a three-year campaign which ran in 2018–19 (year one), 2019–20 (year two) and is planned for 2020–21 (year three). This audit examined year two of the campaign (2019–20).
The Fireproof campaign is a public safety campaign encouraging people to plan and prepare for bush fires across the summer period. The campaign aims to improve the quality of bush fire planning and preparation in the community and decrease the impact of fires on the community when they occur.
Campaign materials we reviewed did not breach section 6 of the Act
The audit team reviewed campaign materials developed as part of the paid advertising campaign for example radio advertisements, television commercials and digital displays. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. Examples of campaign materials are shown in Appendix two.
Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:
- be designed to influence (directly or indirectly) support for a political party
- contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
- contain the name, logo, slogan or any other reference to a political party.
The audit found no breaches of section 6 of the Act in the campaign material we reviewed.
NSW Rural Fire Service did not set targets for the second year of the campaign
The second year of the Fireproof campaign (2019–20) had the same objectives as the first year of the campaign (2018–19), however no specific targets were set for the second year. The advertising submission for the first year of the campaign (2018–19) details the targets for each objective as an increase of ten per cent against the baseline data to be achieved by March 2021, at the end of the three-year campaign.
The second year of the Fireproof campaign (2019–20) was one of the first campaigns approved under the new budget and peer review processes introduced by the Department of Customer Service in 2019–20. The new process for peer review introduced a new template for campaign submissions. The former template for campaign submissions contained more prompts for agencies to ensure the submission contained sufficient detail of campaign objectives, baseline measures, targets, dates for measurement and detail on how they would measure objectives. Despite this, the peer review process should have identified that NSW Rural Fire Service did not set targets for the second year of the campaign.
The 2016 Guidelines for Implementing NSW Government Evaluation Framework for Advertising and Communications requires campaign objectives to be SMART (specific, measurable, achievable, realistic and timed). NSW Rural Fire Service did not meet this requirement for year two of the Fireproof campaign.
Post-campaign evaluations showed increases against four out of five objectives, however there were no specific targets
NSW Rural Fire Service set three campaign objectives at the time it submitted the second year of the campaign (2019–20) to the Department of Customer Service for peer review. However, the post-campaign effectiveness report submitted to the Department of Customer Service measured campaign effectiveness against five campaign objectives. The objectives in the post-campaign effectiveness report were the same objectives set for the first year of the campaign, which is appropriate as this was a repeat campaign.
NSW Rural Fire Service achieved increases against four of their five objectives. However, as noted above there were no specific targets (such as percentage increases) against which performance of the 2019–20 campaign could be measured. Despite this, at the end of the second year, the Fireproof campaign had already achieved some of the targets that NSW Rural Fire Service had set for the end of the third year of the campaign. The post-campaign research showed that both audience recall and exposure to the campaign increased significantly from the prior year. The campaign objectives and results are shown in Exhibit 8.
For those people who already have a bush fire plan, the campaign aimed to increase the number of those plans which have included two or more elements from the Guide to Making a Bush Fire Survival Plan. Elements from the Guide to Making A Bush Fire Survival Plan include actions such as deciding what to take with you if you leave, ensuring you have the right equipment for defending your home and allocating responsibilities to members of a household. The post-campaign evaluation showed that the campaign did not achieve an increase against this objective for people who planned to stay and defend their property rather than leave.
Campaign objectives | Does the post-campaign evaluation show increases against the objective? |
---|---|
1. Continue to increase the number of people that have discussed and/or written a plan with regards to what they will do in the event of a fire. | |
2. Of those who indicate they have a plan, increase the number of people who have included two or more elements from the Guide to Making a Bush Fire Survival Plan: | |
|
|
|
|
3. Increase the frequency in completing preparation activities around a person’s property. | |
4. Increase the number of people who correctly assess it is their responsibility to complete preparation activities and enact their plan without direct intervention from emergency services. | |
5. Visits to MyFirePlan website. |
Key | Yes | No |
NSW Rural Fire Service achieved cost efficiencies by reusing creative content developed in the first year of the campaign
Total creative and production costs incurred in year one of the campaign were $1.08 million. Rather than commissioning new creative materials, NSW Rural Fire Service re-used the same creative content in year two of the campaign. NSW Rural Fire Service incurred $100,000 in creative and production costs in year two of the campaign and achieved cost-efficiencies by reusing the same creative developed in the prior year.
NSW Rural Fire Service allowed sufficient time for cost-efficient media placement and received free media placements
The Department of Customer Service advises agencies to work with media contacts to book media in advance to ensure a cost-efficient placement. Prior to 2019–20, the Department of Premier and Cabinet provided suggested timeframes for agencies to book media as part of the peer review process. For example, it advised agencies to book television six to 12 weeks in advance and book radio advertising two to eight weeks in advance. NSW Rural Fire Service allowed sufficient time for a cost-efficient media placement.
NSW Rural Fire Service received $4 million of free advertising time and space donated by media companies due to the extent and impact of the 2019–20 fire season.
The cost benefit analysis (CBA) did not assess the costs and benefits of alternatives to advertising
Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. As part of the CBA, the Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits.
The CBA for the Fireproof campaign (year two) notes that the proposed campaign is one component of a broader community engagement strategy which has been developed over time and is based on research and evaluation. The CBA considers two options to achieve the objectives of the campaign. The first option is community engagement activities without an advertising campaign and the second option is community engagement activities alongside an advertising campaign. The CBA does not identify and assess the costs and benefits of both of the options in order to assess the most cost-efficient option.
This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.
Appendices
Appendix one – Responses from agencies
Appendix two – About the campaigns
Appendix three – About the audit
Appendix four – Performance auditing
Copyright notice
© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.
Parliamentary reference - Report number #342 - released 19 November 2020