Campaign material did not contain political content and was for an allowable purpose

The audit team reviewed campaign material developed as part of Transport for NSW’s (TfNSW’s) advertising campaign including radio transcripts, digital videos, and display. The audit found no breaches of Section 6 of the Act, which prohibits political content as part of a government advertising campaign.

TfNSW's advertising was also consistent with Section 4 of the Act and Section 1.2 of the NSW Government Advertising Guidelines, which define the nature of, and set out the allowable purposes for, government advertising campaigns.

TfNSW completed a peer review and a draft cost-benefit analysis for the campaign, but it did not comply with the Government Advertising Guidelines

Section 7 of the Act and Section 4 of the Regulation require the head of a government agency to ensure, before the start of the campaign, that:

• a peer review of the proposed campaign is carried out if the cost of the campaign is likely to exceed $250,000
• a cost-benefit analysis is carried out if the cost of the campaign is likely to exceed $1 million.

The peer review is an independent review of the need for the proposed advertising campaign, the creative and media strategy and how the agency will manage the campaign. The Department of Customer Service (DCS) manages the peer review process. TfNSW carried out the peer review in two steps, with an interim peer review for an urgent component of $654,000 on 14 September 2022 and the second non-urgent component of $2.35 million on 27 September 2022. The DCS’ interim peer review for the urgent component allowed TfNSW to commence advertising during the September school holiday period when many road trips are taken.

In August 2022, TfNSW prepared a draft cost-benefit analysis for its investment in the campaign. However, its analysis only considered advertising as a way of achieving the desired objectives of improving road user behaviour and reducing the number of crashes, deaths and serious injuries on New South Wales roads. This analysis therefore did not comply with section 6.4 of the Government Advertising Guidelines which requires that ‘the cost benefit analysis must outline what options other than advertising could be used to successfully implement the program and achieve the program benefits and a comparison of their costs.’ TfNSW advised the audit team that it considered it met this requirement through its inclusion of a 'base case' in the cost-benefit analysis (as the base case involved no advertising taking place). However, a 'base case' cannot be considered an option to implement a program, and no option evaluated by TfNSW met the requirement to consider 'options other than advertising'.

The Advertising Compliance Certificate was not supported by sufficient evidence

Section 8 of the Act requires the head of an agency to certify that the planned campaign:

• complies with the Act, the regulations, and the Guidelines
• contains accurate information
• is necessary to achieve a public purpose and is supported by analysis and research
• is an efficient and cost-effective means of achieving that public purpose.

On 20 October 2022, the TfNSW Secretary signed a compliance certificate for the campaign. The certificate was accompanied by a submission that noted the previous year’s fatalities due to fatigue and stated that an advertising campaign was required to educate drivers on how to avoid driving tired and encourage them to consider how tired they are before driving. The submission included information on channels, media, audiences, creative and budget - which TfNSW told the audit team 'will allow the Secretary to ascertain value'. However, the submission did not provide any evidence about the assessed efficiency or cost-effectiveness of the proposed campaign or address the requirement set out in the Government Advertising Guidelines for agencies to consider options other than advertising to achieve the desired objective.

TfNSW met all other key requirements for a government advertising campaign

NSW Government policy requires agencies to: (a) spend a minimum proportion of their media budget for communications with specific audiences; (b) comply with the NSW Government's evaluation framework and brand guidelines; and (c) ensure advertising is presented in an accurate and objective manner. TfNSW met these requirements in 2022–23.

Government policy also requires that at least 7.5% of an advertising campaign's media budget be spent on direct communications to multicultural and Aboriginal audiences. TfNSW's planned budget for these audiences was 7.62% of their overall media budget and their actual spend was higher at 8.76%.

TfNSW prepared a high-level evaluation framework, consistent with expectations set by DCS in the Evaluation Framework for Advertising and Communications (and associated Guidelines) and section 5.4 of the Government Advertising Guidelines. TfNSW also has a ‘brand hub’ that establishes rules and guidance for the use of TfNSW’s logo and the appropriate presentation of website and other advertising material. The brand hub includes templates that aim to ensure all communications are consistent with the Australian Government’s Web Content Accessibility Guidelines and therefore accessible to people who have disabilities.

The Don’t Trust Your Tired Self (DTYTS) campaign deployed the ‘Towards Zero’ brand. Although this is not a standard NSW Government brand, in December 2022, the Minister for Customer Service and Digital Government approved a three-year temporary exemption from the standard NSW Government brand for the use of the 'Towards Zero' branding.

TfNSW’s campaign complied with the requirement to present information accurately and objectively. The following key facts were each supported by appropriate evidence and data:

• research shows that getting behind the wheel after being awake for about 17 hours affects driving performance similar to a blood alcohol content of 0.05%
• fatigue is one of the three main causes of crashes in New South Wales (the other main causes are speeding and alcohol)
• most fatigue crashes happen close to home.

TfNSW set a clear target audience for the campaign and achieved most of the annual targets, but it has not yet evaluated it’s long-term impact despite running it every year since 2013

The DTYTS campaign was primarily targeted at an audience that TfNSW defined as 'self-confident' or 'change-resistor' motorists. The campaign's secondary audience included all New South Wales motorists.

TfNSW achieved most of the campaign's annual targets, including 13 of 14 output targets and 19 of its 30 outcome targets. The campaign came close to achieving, but did not fully achieve some of the intended output and outcome targets. However, TfNSW has not yet measured the campaign's longer-term contribution to lowering the NSW road toll (this is the campaign's measure of its impact). This is despite the DTYTS campaign being run every year since 2013. While it is reasonable for TfNSW to not measure long-term impacts every year, a decade of campaign delivery provides a clear opportunity for an assessment of the campaign’s long-term impact in New South Wales. TfNSW advised the audit team that evaluating the impact of the campaign on the number of deaths and serious injuries on New South Wales roads is a complex task. The audit team acknowledges the task may be complex but considers that understanding the long-term impact of road safety campaigns is important—particularly as advertising is central to the government's road safety effort. TfNSW has committed to considering this task as part of future evaluations.

TfNSW’s draft cost-benefit analysis for this campaign was not formally finalised

In August 2022, TfNSW prepared a draft cost-benefit analysis for its investment in the campaign. The analysis considered the impact of previous advertising campaigns on the number of road casualties and adjusted the likely impact for the level of campaign expenditure, potential overlapping benefits, and macroeconomic effects such as changes in vehicle technologies. The analysis found that the campaign investment of $3.04 million was expected to generate total benefits of $15.95 million and result in $12.95 million in net benefits (expressed in 2022 dollars). As a result, the campaign was expected to offer a benefit to cost ratio of 5.3 and the draft analysis concluded that investment in the campaign was economically viable. When this audit was conducted, the analysis remained draft and unapproved. However, TfNSW advised that the draft cost-benefit analysis was 'for all purposes, the final document' as no further changes were made. Nonetheless, TfNSW has been unable to locate documentation supporting the finalisation of the cost-benefit analysis.

Breaking the campaign's approval into urgent and non-urgent components helped to address risks to timely advertising but may have risked paying a small cost premium due to buying some media at short notice

In 2022–23, government advertising campaigns costing more than $250,000 were required to be approved by the Expenditure Review Committee of Cabinet. This took place on an annual basis and was coordinated by the DCS Campaign Effectiveness Team. This government process necessarily places some constraints on how much notice agencies have before they can arrange media and commence advertising campaigns – and leads to risks to timely and cost-effective advertising.

TfNSW worked with the DCS Campaign Effectiveness Team to manage risks to timely advertising by separately approving the campaign for urgent and non-urgent components. This allowed the DTYTS advertising to take place in the desired September school holidays (when many road trips are planned and taken). However, TfNSW may have incurred a small cost premium when buying some media at short notice. TfNSW advised the audit team that any such cost premium was outside of their control because of the government approvals process involved.

TfNSW’s spend on this campaign met NSW Government requirements for value for money procurement

TfNSW used three key suppliers to plan and deliver the DTYTS campaign. Each of these suppliers was appointed under competitive arrangements. This is consistent with NSW Government requirements for value for money procurement. Notwithstanding the small premium that TfNSW may have paid for buying some media at short notice (set out in the previous finding), it is therefore likely that the campaign spending overall was economical.

TfNSW cancelled the Saving Lives on Country Roads campaign for 2022–23 around the same time it reported the campaign’s last delivery was successful in many respects

TfNSW last ran the full SLCR campaign in 2021–22, reporting to DCS that the campaign was successful in many respects, including by:

• ‘building strong campaign recognition while consistently raising awareness of the serious consequences of everyday driving decisions on regional New South Wales roads
• being easy to understand, highly believable and personally relevant to regional New South Wales drivers
• shifting regional drivers to be personally committed to safety on country roads’.

The report to DCS also noted that the campaign had been successful in driving desirable attitudes, particularly around the road toll in regional New South Wales being unacceptably high. However, the same report noted that the campaign in 2021–22 had targeted a new primary audience, consisting of three groups that TfNSW defined as 'self-confident' motorists, 'change-resistor' motorists, and 'disconnected' motorists. The results on key campaign metrics (such as the advertisement’s understandability, believability and personal relevance) for this new primary audience were lower than in previous years. Finally, the report to DCS also noted that the campaign had not yet shown any evidence of changed behaviours.

At around the same time as it reported in a mostly positive manner to DCS on the campaign’s delivery in 2021–22, TfNSW cancelled the SLCR campaign for the 2022–23 financial year. Subsequently, in the NSW Road Safety Progress Report 2022 (tabled in Parliament and published in November 2023), TfNSW reported publicly on the positive effects of the SLCR campaign and made no reference to its decision to cancel the campaign following its last delivery in 2021–22.

TfNSW told the audit team that the progress report 'refers to the previous year and not to the period in which its effectiveness declined' and the period 'in which the decision to cancel the campaign had not been made'. However, the progress report is on road safety for the 2022 calendar year and SLCR was cancelled in this period. Accordingly, the progress report should have included the cancellation of SLCR and described the reasons for its cancellation.

At the time of this audit, TfNSW told the audit team that the overall measure of a campaign's effectiveness is its impact on behaviour. TfNSW also stated that as it believed the SLCR campaign was not impacting behaviour, it was replaced with behavioural campaigns that it believed were more effective in changing driver behaviour.

TfNSW could have improved the formal documentation of its decision-making process when it cancelled the Saving Lives on Country Roads campaign

TfNSW advised the audit team that its rationale for cancelling the campaign was because it was particularly concerned that the campaign had declining results for personal and social relevance (that is, audience feedback about whether the advertisement was ‘for people like me’ and whether the advertisement would encourage them to ‘talk to other people about it’). TfNSW also advised the audit team that the cancellation of the campaign relied upon evidence from a private sector firm with specialist expertise in the field, and that it discussed the cancellation of the campaign with the Centre for Road Safety (a part of TfNSW).

TfNSW advised the audit team late in the audit process that a review of the value of the SLCR campaign in light of the portfolio of campaigns did take place. It also advised the audit team that it discussed the cancellation of SLCR with DCS. However, TfNSW did not provide the audit team with any documentary evidence supporting either of these statements.

TfNSW could have prepared and documented a more comprehensive evidence base for its decision to cancel the campaign. For example, it could have:

• documented the campaign’s overall effectiveness, options to redevelop it, and evidence for the cancellation
• documented the proposed cancellation’s effects on the overall portfolio of TfNSW campaigns. For example, the campaign may have been supporting and reinforcing safety messages in other complementary TfNSW campaigns
• considered the cancellation of the campaign in a formal governance forum
• documented the rationale or overall decision-making process, including the decision-maker, for deciding to cancel the campaign.

Cancellation of the Saving Lives on Country Roads campaign meant that TfNSW relied upon the state-wide advertising campaigns for addressing road safety in regional NSW

The SLCR campaign aimed to encourage country drivers to re-think the common excuses used to justify their behaviour on the road.

TfNSW continues to run advertising campaigns covering regional New South Wales which aim to change specific driver attitudes or behaviours (such as drink driving). TfNSW advised the audit team that these campaigns either depict regional settings, are bespoke for regional audiences, or have tested positively with regional audiences.

TfNSW further advised the audit team that it spent $6.33 million on media advertising to regional audiences in 2022–23 across all road safety advertising campaigns (representing 34.78% of its actual spending). While this was a relatively small reduction on the $6.59 million it spent on media advertising in the prior year (36.09% of its actual spending), TfNSW told the audit team that it is confident that it has maintained the presence of its campaigns in regional media.

¹ The words ‘country’ and ‘regional’ are used interchangeably throughout this report, reflecting Transport for NSW’s use of these terms in documents reviewed by the Audit Office of New South Wales.

By 31 October 2024, TfNSW should implement processes that ensure:

1. cost-benefit analyses prepared for government advertising campaigns comply with the Government Advertising Guidelines
2. long-term impacts of advertising campaigns are evaluated
3. strategic and operational decision-making about advertising campaigns, such as starting, stopping or significantly changing a campaign, is well-documented and follows good practice.

What is government advertising?

Governments use advertising to communicate information about a government program, policy or initiative, or about any public health or safety or other matter, to members of the public. In New South Wales, the Department of Customer Service (DCS) states that an activity is considered 'government advertising' if it is funded by or on behalf of a government agency, and is disseminated under a commercial advertising distribution agreement through a variety of media, such as radio, television, the Internet, newspapers, billboards or cinemas.

There are three broad categories of government advertising:

• recruitment advertising - promotes specific job vacancies and employment opportunities within a government agency
• public notices - communicates a clear, simple message or announcement and is generally one-off or short-term in nature
• public awareness advertising - co-ordinated communications to raise awareness of key issues, such as government initiatives, or to encourage behaviour change.

How much is spent on government advertising in New South Wales?

Section 3 of the Government Advertising Act 2011 states that the 'cost of a government advertising campaign includes the amount paid or payable by or on behalf of a government agency for the following:

a) research for the purposes of the campaign
b) production or carrying out of the campaign
c) media distribution
d) evaluation of the likely or actual effectiveness of the campaign’.

The NSW Government does not publish a consolidated report on the full cost of government advertising as defined above, and there is no statutory requirement to do so. Every six months, DCS publishes details of advertising media expenditure by government agencies (a subset of the full cost), consisting of payments to media organisations for running advertisements. DCS also encourages individual agencies to publish information about advertising programs on their own websites. Information may include advertising rationale, objectives, costs and outcomes.

In 2022–23, the NSW Government spent $131.46 million on media advertising. Exhibit 1 shows government expenditure on media advertising for the financial years 2015–16 to 2022–23.

Transport for NSW (TfNSW) is the NSW Government agency responsible for leading the development of safe, integrated and efficient transport systems for the people of New South Wales. TfNSW was established in November 2011 and is the principal agency of the Transport, Roads, and Regional Transport and Roads portfolios.

At the end of 2022–23, TfNSW employed 15,594 people and had total expenses of $24.7 billion.

The Centre for Road Safety is a part of TfNSW which aims to develop practical solutions to reduce the number of deaths and serious injuries on New South Wales roads. The Centre helps to deliver road safety strategies, advertising campaigns, collaboration with the community, and approaches to designing roads and vehicles which can make a significant impact on the outcome of road crashes in New South Wales.

TfNSW carries out a variety of advertising campaigns every year. Information on campaigns relating to road safety may be found in the NSW Road Safety Progress Report published annually by TfNSW.

The 2022 NSW Road Safety Progress Report (published in November 2023), the latest report available, showed that there was total expenditure of $27.2 million on road safety campaigns in 2021–22, including advertising, research, partnerships, and engagement.

This performance audit originally intended to examine two campaigns run by the Centre for Road Safety NSW (a part of TfNSW) in 2022–23:

• Don’t Trust Your Tired Self (DTYTS), a state-wide campaign, which aims to educate drivers on how to avoid driving tired and encourage them to consider how tired they are before driving. The campaign was shown on television, digital video, radio, outdoor advertising, social media and online.
• Saving Lives on Country Roads (SLCR), a regional campaign, which aimed to encourage country drivers to re-think the common excuses used to justify their behaviour on the road.

In early 2024, following advice provided by DCS to the audit team, TfNSW confirmed that the SLCR campaign was not run during 2022–23 as the campaign ceased after delivery in 2021–22.

As a result, this audit focused on TfNSW’s advertising campaign for DTYTS. The campaign had a planned advertising budget of $3 million and spent $3.04 million.

TfNSW established the DTYTS campaign to:

• positively impact understanding and attitudes of the risks of driving tired
• positively impact drivers' intentions to:
  • get a good night’s sleep before driving
  • (never) drive or ride if they need to get somewhere but feel like they might be too tired to drive or ride
  • stop and take a break or swap drivers if they are driving and start to feel like they might be too tired to drive.

TfNSW supported the campaign with its website TestYourTiredSelf.com.au which allows motorists to take an interactive test to see how tired they might be before driving.
In its 2022 NSW Road Safety Progress Report, TfNSW stated that: ‘In March 2022 a new commercial was launched on online video and social media to highlight the risk fatigue poses on everyday driving. Sixty-one per cent of the target audience recognised the campaign, with 60% of those who had seen it stating they will plan to get a good night’s sleep before driving’.

The audit briefly examined the reasons for the cancellation of the SLCR campaign (its expenditure in 2021–22 was $3.11 million with $17,038 spent in 2022–23). This included reviewing the:

• decision-making process for the campaign’s cancellation
• rationale and evidence for the campaign’s cancellation.

In its 2022 NSW Road Safety Progress Report, TfNSW stated that:

Appendix two contains further detail about these two TfNSW advertising campaigns.

Section 14 of the Act requires the Auditor-General for New South Wales to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year.

In conducting the audit, the Auditor-General must determine whether a government agency (or agencies) has carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the Guidelines and other laws. Appendix three contains further details about this audit.

Campaign material did not contain political content

The audit team reviewed campaign material developed as part of TfNSW’s advertising campaign including radio transcripts, digital videos, and display.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of Section 6 of the Act in the campaign material reviewed.

Campaign material was for an allowable purpose

Section 4 of the Act states that government advertising campaigns are for ‘the dissemination to members of the public of information about a government program, policy or initiative, or about any public health or safety or other matter’. To support this, section 1.2 of the NSW Government Advertising Guidelines states that government advertising campaigns may only be used to achieve certain objectives, one of which is to:

• encourage changed behaviours or attitudes that will lead to improved public health and safety or quality of life.

The audit team considers that the DTYTS campaign was consistent with the Act and the supporting Guidelines. This reflects the campaign intent to improve road user behaviour and reduce the number of crashes, deaths and serious injuries on New South Wales roads by educating drivers to:

• understand the risks of driving tired
• get a good night’s sleep before driving
• identify the signs of fatigue when driving
• take a break or share the drive if they are tired.

TfNSW completed a peer review for the campaign in two steps, covering an initial urgent component and then the remaining non-urgent component

Section 7 of the Act and the supporting Regulation require the head of a government agency to ensure, before the start of the campaign, that a peer review of the proposed campaign is carried out if the cost of the campaign is likely to exceed $250,000.

The peer review is an independent review of the need for the proposed advertising campaign, the creative and media strategy (including objectives and target audiences) and how the agency will manage the campaign. The Department of Customer Service (DCS) manages the peer review process.

TfNSW carried out the peer review for the DTYTS campaign in two steps, with an interim peer review for an urgent component of $654,000 completed by DCS on 14 September 2022 and the second non-urgent component of $2,346,000 completed by DCS on 27 September 2022. The DCS’ interim peer review for the urgent component allowed TfNSW to commence advertising during the September school holiday period when many road trips are taken.

TfNSW completed a draft cost-benefit analysis for the campaign, but it did not comply with the Government Advertising Guidelines

Section 7 of the Act and the supporting Government Advertising Guidelines require the head of a government agency to ensure, before the start of the campaign, that a cost-benefit analysis is carried out if the cost of the campaign is likely to exceed $1 million.

The Guidelines require that ‘the cost benefit analysis must outline what options other than advertising could be used to successfully implement the program and achieve the program benefits and a comparison of their costs'.

In August 2022, TfNSW prepared a draft cost-benefit analysis for its investment in the DTYTS campaign. However, the analysis only considered advertising as a way of achieving the desired objectives of improving road user behaviour and reducing the number of crashes, deaths and serious injuries on New South Wales roads. TfNSW therefore did not comply with the requirement set out in section 6.4 of the Government Advertising Guidelines to consider options other than advertising to achieve desired objectives. For example, TfNSW could have considered options such as contracting with other organisations or changing its sponsorship profiles or designing other interventions (potentially in combination with the same, more, or less advertising) to achieve the desired attitudinal and behaviour changes.

TfNSW advised the audit team that it 'took the position that the Base Case option accommodates this ‘no advertising option’ requirement because the Base option involved no advertising expenditure taking place'. NSW Treasury guidance² is that a base case is used as 'a comparator to the initiative options' which allows the benefits and costs of all options to be calculated relative to the base case. The audit team considers that using a base case or 'do nothing' scenario as an implementation 'option' is inconsistent with NSW Treasury guidance, and does not allow for genuine assessment of alternative investment options against the preferred course of action. No option evaluated by TfNSW met the requirement to consider 'options other than advertising'.

TfNSW’s cost-benefit analysis was completed but not formally approved. It remained in draft form when this audit was conducted. TfNSW advised the audit team that the draft document was 'for all purposes, the final document'. TfNSW also advised that it has been unable to locate documentation supporting the finalisation of the cost-benefit analysis.

The Advertising Compliance Certificate was not supported by sufficient evidence

Under the Act, the head of an agency must sign a compliance certificate before commencing the campaign. The compliance certificate is required by Section 8 of the Act and states that the head of the agency confirms that a proposed government advertising campaign:

• complies with the Act, the regulations, and the Guidelines
• contains accurate information
• is necessary to achieve a public purpose and is supported by analysis and research
• is an efficient and cost-effective means of achieving that public purpose.

On 20 October 2022, the TfNSW Secretary signed a compliance certificate for the DTYTS campaign. The certificate was accompanied by a submission that noted the previous year’s fatalities due to fatigue and stated that an advertising campaign was required to educate drivers on how to avoid driving tired and encourage them to consider how tired they are before driving. The submission included information on channels, media, audiences, creative and budget which TfNSW told the audit team would 'allow the Secretary to ascertain value'. The submission also noted that the campaign had been reviewed by the DCS peer review process but, other than that, it did not:

• provide evidence about the assessed efficiency or cost-effectiveness of the proposed campaign (for example by including the results of the cost-benefit analysis)
• address the requirement set out in the Government Advertising Guidelines for agencies to consider options other than advertising to achieve the desire objective.

² NSW Government Guide to Cost-Benefit Analysis. TPG23-08. February 2023.

TfNSW met the recommended minimum spend of their media budget for communications with Culturally and Linguistically Diverse (CALD) and Aboriginal audiences

NSW Government policy requires that government advertising be sensitive to cultural needs and issues and reflect the cultural and linguistic diversity of New South Wales. It also requires that at least 7.5% of an advertising campaign media budget be spent on direct communications to multicultural and Aboriginal audiences. Spend may be on media or non-media communication activities (for example, events, participation at cultural festivals, direct mail, competitions, and websites).

For the DTYTS campaign, TfNSW’s planned media budget for direct communications was $2.544 million, of which $193,798 was intended for media directly targeted to multicultural and Aboriginal audiences. This represented a budget for CALD and Aboriginal audiences of 7.62%. TfNSW’s actual spend on advertising to these audiences was $216,271 and its actual overall spend on advertising was $2.470 million. As a result, TfNSW spent 8.76% of its total on advertising to multicultural and Aboriginal audiences.

The campaign complied with the NSW Government’s evaluation framework and brand guidelines

TfNSW prepared a high-level evaluation framework, which was consistent with expectations set by DCS and section 5.4 of the Government Advertising Guidelines.

TfNSW also has a ‘brand hub’ that establishes rules and guidance for the use of TfNSW’s logo and the appropriate presentation of website and other advertising material. The brand hub includes templates that aim to ensure all communications are consistent with Web Content Accessibility Guidelines and therefore accessible to people who have disabilities. The DTYTS campaign deployed the ‘Towards Zero’ brand. Although this is not a standard NSW Government brand, in December 2022 the Minister for Customer Service and Digital Government approved a three-year temporary exemption from the standard NSW Government brand for the use of the ‘Towards Zero’ branding.

TfNSW’s advertising was presented in an accurate and objective manner

The Government Advertising Guidelines require that all facts, statistics, comparisons, and other arguments are presented accurately, and all statement and claims of fact in government advertising must be able to be substantiated. They further require that government advertising must be objectively presented in a fair and accessible manner.

TfNSW’s DTYTS campaign includes the following key facts, each of which is supported by appropriate evidence and data:

• research shows that getting behind the wheel after being awake for about 17 hours affects driving performance similar to a blood alcohol content of 0.05%
• fatigue is one of the three main causes of crashes in New South Wales (the other main causes are speeding and alcohol)
• most fatigue crashes happen close to home.

Accordingly, TfNSW’s campaign complied with the requirement to present information accurately and objectively.

TfNSW set a clear target audience for the campaign

The Government Advertising Guidelines require that advertising campaigns be evaluated on key factors including whether the target audience is ‘well defined and understood’.

TfNSW’s DTYTS campaign had two key target audiences:

• a primary audience, that TfNSW defined as 'self-confident' or 'change-resistor' motorists
• a secondary audience, which included all New South Wales motorists.

TfNSW defined 'self-confident' and 'change-resistor' motorists using a range of demographic and behavioural characteristics including their:

• sex, both groups have more males than the general population
• age, both groups tend towards those aged between 30 and 59
• attitude to risk, both groups are more likely to behave in risky ways
• willingness to change and preferences for receiving/consuming advertising or other communications.

TfNSW’s audience segmentation was supported by detailed research carried out by their research and evaluation partner, a private sector firm with specialist expertise in the field. The same audience segmentation was then used as part of TfNSW’s campaign effectiveness reports provided to the Department of Customer Service (DCS).

TfNSW established relevant objectives and measures for the campaign

The NSW Government ‘Evaluation Framework for Advertising and Communications’ (and associated Guidelines) requires advertising campaigns to be measured against their:

• Outputs, or what is done that reaches and engages the target audience. Outputs are assessed using metrics of exposure and reception.
• Outcomes, or what the target audience takes from the communication and what sustainable effect the communication has on the audience. Outcomes are measured by the consumer response to the campaign.
• Impacts, or the results that are caused by the campaign. Impacts are measured by objective actions or events.

TfNSW complied with the Evaluation Framework. It prepared a ‘Campaign Objectives Table’ which sets out detailed information on baseline and targeted performance, measurement milestones and evaluation methods. Measures can be evidenced and are linked to the campaign objectives. Using information collected by a private sector firm with specialist expertise in the field, TfNSW reviews the campaign's performance during and following the implementation of the campaign. Exhibit 4 summarises TfNSW’s measurement of its annual campaign.

Source: Audit Office of New South Wales analysis of TfNSW evaluation documentation.

TfNSW achieved most of the campaign’s annual targets but has not yet evaluated its long-term impact, despite running it every year since 2013

The DTYTS campaign met 13 of 14 output targets and 19 of 30 outcome targets. The campaign only narrowly did not achieve some of the intended output and outcome targets.

TfNSW demonstrated that it used a structured process to track the campaign’s performance. However, it has not yet measured the campaign's longer-term contribution to lowering the road toll in New South Wales (the impact measure). This is despite DTYTS being run every year since 2013. While it is reasonable for TfNSW to not measure long-term impacts after each year of delivery, a decade of campaign delivery provides a clear opportunity for correlational or other assessment of the campaign’s long-term impact.

Exhibit 5 summarises the campaign’s achievements against planned outputs, outcomes, and impacts.

Source: Audit Office of New South Wales analysis of TfNSW evaluation documentation.

TfNSW’s draft cost-benefit analysis for this campaign was not formally finalised

In August 2022, TfNSW prepared a draft cost-benefit analysis for its investment in the DTYTS advertising campaign. The analysis considered the impact of previous advertising campaigns on the number of road casualties and adjusted the likely impact for:

• campaign expenditures
• potential overlapping benefits
• macroeconomic effects such as changes in vehicle technologies.

TfNSW’s analysis found that the campaign investment of $3 million was expected to generate total benefits of $15.954 million and result in $12.954 million in net benefits (expressed in 2022 dollars). As a result, the campaign was expected to offer a benefit to cost ratio of 5.3. In other words, the economic benefit arising from the campaign was 5.3 times its cost. TfNSW’s draft analysis therefore concluded that investment in the campaign was economically viable.

TfNSW’s cost-benefit analysis was not approved and remained in draft form when this audit was conducted. TfNSW advised the audit team that the draft document was 'for all purposes, the final document'. TfNSW also advised that the report was reviewed by their Finance and Investment Division and no further work was required. Nonetheless, TfNSW has been unable to locate documentation supporting the finalisation of the cost-benefit analysis.

TfNSW supported this campaign with its long-standing 'Test Your Tired Self' website

According to TfNSW, driving when tired is one of the top three killers on New South Wales roads, along with speeding and drink driving. Fatigue crashes are also three times as likely to be fatal than other crashes.

In 2013, TfNSW first launched a road safety campaign targeting fatigue. As part of that campaign, the website TestYourTiredSelf.com.au was launched with the Minister for Roads and Ports describing the website as a:

TfNSW has continued to use the website to support its fatigue-based advertising campaigns. All motorists can take the interactive test to see how tired they might be before driving. Over the five years from July 2018 to June 2023, 173,823 tests were completed on the website. As at June 2023, there were 5,908,876 active licence holders in New South Wales. TfNSW advised the audit team that this website cost $14,878 to host and maintain in 2022–23.

Breaking the campaign’s approval into urgent and non-urgent components helped to address risks to timely advertising but may have risked paying a small cost premium due to buying some media at short notice

In 2022, government advertising campaigns costing more than $250,000 required approval by the Expenditure Review Committee of Cabinet. This took place on an annual basis and was coordinated by the DCS Campaign Effectiveness Team. This government process necessarily placed some constraints on how much notice agencies have before they can arrange media and commence advertising campaigns – and led to risks to timely and cost-effective advertising.

TfNSW worked with the DCS Campaign Effectiveness Team to manage risks to timely advertising by separately approving the campaign for urgent and non-urgent components. This allowed DTYTS advertising to take place in the desired September school holidays (when many road trips are planned and taken).

However, TfNSW may have incurred a small cost premium due to buying some media at short notice. This is particularly likely to have affected TfNSW’s television advertising. While TfNSW has not estimated the value of any cost premium, information provided as part of the tender for the whole-of-government Media Agency Services Contract in 2022 showed that media bought at short notice was likely to attract a cost premium of between four per cent and 28% (depending on the length of the notice before the buying takes place). Using this information, the Audit Office considers that TfNSW is likely to have paid a cost premium on its media bought in September and October 2023. The premium is likely to have been at least $50,000.

TfNSW advised the audit team that any such cost premium was outside their control because of the government approvals process involved.

TfNSW’s spend on this campaign met NSW Government requirements for value for money procurement

TfNSW spent $3.04 million against its planned $3 million budget and used three key suppliers to plan and deliver the DTYTS campaign. These were a:

• Private sector firm with specialist expertise in the field to provide insight into citizen and road user views. This supplier cost $90,800 or three per cent of the spend on this campaign.
• Creative agency to develop content for use in advertising campaigns. The creative content for this campaign cost $167,328 or six per cent of the spend.
• Media partner to plan, buy and execute the media component of advertising campaigns. This included $276,184 or nine per cent of the campaign spend for planning and strategy services, and $2,469,852 or 82% of the spend on media buying and executing. 

Each of TfNSW’s key suppliers to this campaign were appointed under competitive arrangements. This is consistent with NSW Government requirements for value for money procurement. Notwithstanding the small premium that TfNSW is likely to have paid for buying some media at short notice (see the previous finding), it is likely that the campaign spending overall was economical.

TfNSW cancelled the Saving Lives on Country Roads campaign for 2022–23 around the same time it reported the campaign’s last delivery was successful in many respects

TfNSW last ran the SLCR campaign in 2021–22, reporting to the Department of Customer Service (DCS) that the campaign was successful in many respects, including by:

• building strong campaign recognition while consistently raising awareness of the serious consequences of everyday driving decisions on regional New South Wales roads
• being easy to understand, highly believable and personally relevant to regional New South Wales drivers
• shifting regional drivers to be personally committed to safety on country roads.

The report to DCS also noted that the campaign had been successful in driving desirable attitudes, particularly around the road toll in regional New South Wales being unacceptably high. However, the same report noted that the campaign in 2021–22 had moved away from the previous years’ focus on all New South Wales drivers and instead had targeted a new primary audience, consisting of three groups that TfNSW defined as 'self-confident' motorists, 'change-resistor' motorists, and 'disconnected' motorists. The results on key campaign metrics (such as the advertisement’s understandability, believability and personal relevance) for this new primary audience were lower than in previous years. These lower results were consistent with concerns raised by DCS on the campaign’s planned effectiveness when it approved it at the start of the year. Finally, the report to DCS noted that the current phase of the campaign had focused on encouraging behaviour change (and not simply attitudinal change) in the key area of drink driving but that it had not yet shown any evidence of changed behaviours.

At around the same time as it reported in a mostly positive manner to DCS on the campaign’s delivery in 2021–22, TfNSW decided to cancel the SLCR campaign.

Subsequently, in the NSW Road Safety Progress Report 2022 (tabled in Parliament and published in November 2023), TfNSW reported positive effects of the SLCR campaign and made no reference to its decision to cancel the campaign following its last delivery in 2021–22. TfNSW told the audit team that the November 2023 Progress Report 'refers to the previous year and not to the period in which its effectiveness declined' and the period 'in which the decision to cancel the campaign had not been made'. However, the NSW Road Safety Progress Report 2022 is a report on road safety for the 2022 calendar year and SLCR was cancelled in this period. Accordingly, the NSW Road Safety Report could have included the cancellation of SLCR and described the reasons for its cancellation.

At the time of this audit, TfNSW informed the audit team that the overall measure of a campaign's effectiveness is its impact on behaviour. TfNSW also stated that as it believed the SLCR campaign was not impacting behaviour, the campaign was replaced with behavioural campaigns that it believed were more effective in changing driver behaviour.

TfNSW could have improved the formal documentation of its decision-making process when it cancelled the Saving Lives on Country Roads campaign

At the end of 2021–22, TfNSW decided that it would stop running the SLCR campaign. TfNSW advised the audit team that it could not locate any documentation to support its decision to cancel the campaign but its rationale was that it was particularly concerned that the campaign had declining results for personal and social relevance (that is, audience feedback about whether the advertisement was ‘for people like me’ and whether the advertisement would encourage me to ‘talk to other people about it’). TfNSW also advised the audit team that it considered evidence from its research and evaluation partner, a private sector firm with specialist expertise in the field, and that it discussed the cancellation with the Centre for Road Safety (a part of TfNSW).

Stopping this campaign was an operational decision for TfNSW. However, when TfNSW cancelled the campaign, it could have prepared and documented a more comprehensive evidence base for this decision. For example, it could have:

• documented the campaign’s overall effectiveness, options to redevelop it, and evidence for the cancellation
• documented the proposed cancellation’s effects on the overall portfolio of TfNSW campaigns. For example, the campaign may have been supporting and reinforcing safety messages in other complementary TfNSW campaigns
• considered the cancellation of the campaign in a formal governance forum
• documented the rationale or overall decision-making process, including the decision-maker, for deciding to cancel the campaign.

TfNSW advised the audit team late in the audit process that a review of the value of the SLCR campaign in light of the portfolio of campaigns did take place. However, TfNSW did not provide any documentary evidence supporting this.

The Saving Lives on Country Roads campaign was cancelled with limited formal documentation of the decision-making process. Instead, TfNSW now relies on state-wide advertising campaigns with dedicated regional components.

The SLCR campaign aimed to encourage country drivers to re-think the collection of common excuses used to justify their behaviour on the road.

TfNSW continues to run advertising campaigns covering regional New South Wales, which aim to change specific driver attitudes or behaviours (such as drink driving). TfNSW advised the audit team that these campaigns either depict regional settings, are bespoke for regional audiences, or have tested positively with regional audiences.

TfNSW also advised the audit team that it spent $6.33 million on media advertising to regional audiences in 2022–23 across all road safety advertising campaigns (representing 34.78% of its actual spending). While this was a relatively small reduction on the $6.59 million spent on media advertising in the prior year (36.09% of its actual spending), TfNSW told the audit team that it is confident it has maintained the presence of its campaigns in regional media.

In 2024, two TfNSW’s websites incorrectly showed Saving Lives on Country Roads as a current campaign, despite the campaign last running in 2021–22

As at 3 June 2024, the TfNSW’s main website and a separate TfNSW site dedicated to the Towards Zero campaign³ both included the SLCR advertising campaign. However, this campaign ceased more than 19 months earlier in 2021–22.

TfNSW told the audit team that these websites provide important resources that can be reused or remain useful after the close of a campaign. This point is acknowledged. However, the utility of a public resource such as this would not be undermined by clearly stating that the campaign was previously run by TfNSW but is no longer in market. This would also help promote transparency and clarity about which campaigns TfNSW runs at any point in time.

³ Towards Zero also includes other activity and significant infrastructure investment but the website Saving Lives on Country Roads | Towards Zero (nsw.gov.au) includes the marketing campaign Saving Lives on Country Roads.