This report analyses the internal controls and governance of the 25 largest agencies in the NSW public sector, excluding state owned corporations and public financial corporations, for the year ended 30 June 2021.

Our preferred approach is to table the ‘Report on State Finances’ in Parliament before any other cluster report. This is because the ‘Report on State Finances’ focuses on the audit results and observations relating to the Total State Sector Accounts, in effect a consolidation of all government agencies. This year the ‘Report on State Finances’ has been delayed due to significant accounting issues being considered in the Total State Sector Accounts and which may impact the Treasury and Transport clusters.

As there are no matters in this report impacting the Total State Sector Accounts we have decided to break with normal practice and table this report ahead of the ‘Report on State Finances’.

What the report is about

This report analyses the internal controls and governance of the 25 largest agencies in the NSW public sector, excluding state owned corporations and public financial corporations, for the year ended 30 June 2021.

What we found

Internal control trends

The proportion of control deficiencies identified as high risk this year increased to 2.8 per cent (2.5 per cent in 2019–20). Six high risk findings related to financial controls while three related to IT controls. Two were repeat findings from the previous year.

Repeat findings of control deficiencies now represent 49 per cent of all findings (42 per cent in 2019–20).

Information technology

We continue to see a high number of deficiencies relating to IT general controls, particularly around user access administration and privileged user access which affected 82 per cent of agencies.

Cyber security

Agencies' self-assessed maturity levels against the NSW Cyber Security Policy (CSP) mandatory requirements are low. Although agencies are required to demonstrate continuous improvement against the CSP, 20 per cent have not set target levels and of those that have set target levels, 40 per cent have not met their target levels.

Policies, processes and definition around security incidents and data breaches lack consistency. Improvement is required to ensure breaches are recorded in registers and action taken to address the root cause of incidents.

Conflicts of interest

Agencies' policies generally meet the minimum requirements of the Ethical Framework set out in the Government Sector Employment Act 2013. However, few meet the Independent Commission Against Corruption's best practice guidelines. Policies could be strengthened in relation to requirements around annual declarations of interests from employees and contractors.

Masterfile management

Policies governing the management of supplier masterfiles and employee masterfiles existed in 79 per cent and 54 per cent of agencies respectively.

Weaknesses were identified in those policies. Access restriction, segregation of duties and record keeping were the most common opportunities for improvement.

Tracking recommendations

Most agencies do not maintain a register to monitor recommendations from performance audits and public inquiries. Registers of recommendations could be improved to include risk ratings and record revisions to due dates. While recommendations can take several years to fully address, the oldest open items were originally due for completion by June 2016.

What we recommended

Agencies should:

• prioritise actions to address repeat control deficiencies, particularly those that have been repeated findings for a number of years
• prioritise improvements to their cyber security and resilience as a matter of urgency
• formalise and implement policies on tracking and monitoring the progress of implementing recommendations from performance audits and public inquiries.

Internal controls are processes, policies and procedures that help agencies to:

• operate effectively and efficiently
• produce reliable financial reports
• comply with laws and regulations
• support ethical government.

This chapter outlines the overall trends for agency controls and governance issues, including the number of audit findings, the degree of risk those deficiencies pose to the agency, and a summary of the most common deficiencies we found across agencies. The rest of this report presents this year’s controls and governance findings in more detail.

The scope of this year's report covers 25 general government sector agencies. Last year's report covered 40 agencies within the total state sector. For consistency and comparability, we have adjusted the 2020 results to include only the agencies remaining within scope of this year's report. Therefore, the 2020 figures will not necessarily align with those reported in our 2020 report.

This chapter outlines our audit observations, conclusions and recommendations arising from our review of agency controls to manage key financial systems.

This chapter outlines our audit observations, conclusions and recommendations arising from our review of agencies' cyber security planning and governance arrangements.

This chapter outlines our audit observations, conclusions and recommendations arising from our review of agencies' conflicts of interest management processes.

This chapter outlines our audit observations, conclusions and recommendations arising from our review of agency's management of supplier and employee masterfiles.

This chapter outlines our audit observations, conclusions and recommendations arising from our review of agencies' processes to track and monitor the implementation of recommendations from performance audits and public inquiries.

What the report is about

The term ‘machinery of government’ refers to the way government functions and responsibilities are organised.

The decision to make machinery of government changes is made by the Premier. Changes may be made for a range of reasons, including to support the policy and/or political objectives of the government of the day.

Larger machinery of government changes typically occur after an election or a change of Premier.

This report assessed how effectively the Department of Planning, Industry and Environment (DPIE) and the Department of Regional NSW (DRNSW) managed their 2019 and 2020 machinery of government changes, respectively. It also considered the role of the Department of Premier and Cabinet (DPC) and NSW Treasury in overseeing machinery of government changes.

What we found

The anticipated benefits of the changes were not articulated in sufficient detail and the achievement of benefits has not been monitored. The costs of the changes were not tracked or reported.

DPC and NSW Treasury provided principles to guide implementation but did not require departments to collect or report information about the benefits or costs of the changes.

The implementation of the machinery of government changes was completed within the set timeframes, and operations for the new departments commenced as scheduled.

Major implementation challenges included negotiation about the allocation of corporate support staff and the integration of complex corporate and ICT systems.

What we recommended

DPC and NSW Treasury should:

• consolidate existing guidance on machinery of government changes into a single document that is available to all departments and agencies
• provide guidance for departments and agencies to use when negotiating corporate services staff transfers as a part of machinery of government changes, including a standard rate for calculating corporate services requirements
• progress work to develop and implement common processes and systems for corporate services in order to support more efficient movement of staff between departments and agencies.

The term ‘machinery of government’ refers to the way government functions and responsibilities are allocated and structured across government departments and agencies. A machinery of government change is the reorganisation of these structures. This can involve establishing, merging or abolishing departments and agencies and transferring functions and responsibilities from one department or agency to another.

The decision to make machinery of government changes is made by the Premier. These changes may be made for a range of reasons, including to support the policy and/or political objectives of the government of the day. Machinery of government changes are formally set out in Administrative Arrangements Orders, which are prepared by the Department of Premier and Cabinet, as instructed by the Premier, and issued as legislative instruments under the Constitution Act 1902.

The heads of agencies subject to machinery of government changes are responsible for implementing them. For more complex changes, central agencies are also involved in providing guidance and monitoring progress.

The NSW Government announced major machinery of government changes after the 2019 state government election. These changes took place between April and June 2019 and involved abolishing five departments (Industry; Planning and Environment; Family and Community Services; Justice; and Finance, Services and Innovation) and creating three new departments (Planning, Industry and Environment; Communities and Justice; and Customer Service). This also resulted in changes to the 'clusters' associated with departments. The NSW Government uses clusters to group certain agencies and entities with related departments for administrative and financial management. Clusters do not have legal status. Most other departments that were not abolished had some functions added or removed as a part of these machinery of government changes. For example, the functions relating to regional policy and service delivery in the Department of Premier and Cabinet were moved to the new Department of Planning, Industry and Environment.

Our Report on State Finances 2019, tabled in October 2019, outlined these changes and identified several issues that can arise from machinery of government changes if risks are not identified early and properly managed. These include: challenges measuring the costs and benefits of machinery of government changes; disruption to services due to unclear roles and responsibilities; and disruption to control environments due to staff, system and process changes.

In April 2020, the Department of Regional NSW was created in a separate machinery of government change. This involved moving functions and agencies related to regional policy and service delivery from the Department of Planning, Industry and Environment into a standalone department.

This audit assessed how effectively the Department of Planning, Industry and Environment (DPIE) and the Department of Regional NSW (DRNSW) managed their 2019 and 2020 machinery of government changes, respectively. It also considered the role of the Department of Premier and Cabinet and NSW Treasury in overseeing machinery of government changes. The audit investigated whether:

• DPIE and DRNSW have integrated new responsibilities and functions in an effective and timely manner
• DPIE and DRNSW can demonstrate the costs of the machinery of government changes
• The machinery of government changes have achieved or are achieving intended outcomes and benefits.

Appendix one – Response from agencies

Appendix two – About the audit

Appendix three – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #359 - released (17 December 2021).

What the report is about

The Auditor-General's review analyses claims made by members of the NSW Parliament during the 2020–21 financial year by testing a sample of transactions. Our sample consisted of 67 claims submitted by 52 of the 137 members.

What we found

While we did not identify any instances of material non-compliance with the Parliamentary Remuneration Tribunal's Determination, we did identify 31 departures from the Determination, which were of an administrative nature.

What we recommended

The Department of Parliamentary Services (the department) should continue to work with the Presiding Officers, members, the Clerk of the Parliaments and the Clerk of the Legislative Assembly to enhance reporting of members' expenditure.

In 2020, we recommended the department work with the Tribunal to provide additional guidance to members to clarify:

• the definition of 'parliamentary duties'
• the activities that meet the definition
• requirements for retaining documents.

The department will work with the Tribunal to clarify these items as part of its submission to the 2022 annual Determination.

The Auditor-General has reviewed the compliance of the members of the NSW Parliament (members) with certain requirements outlined in the Parliamentary Remuneration Tribunal's Determination (the Determination) for the year ended 30 June 2021.

The Auditor-General's review analyses claims made by members during the 2020–21 financial year by testing a sample of transactions. Our sample consisted of 67 claims submitted by 52 of the 137 members.

Results

Although our review did not identify any instances of material non-compliance with the Determination for the year ended 30 June 2021, we did identify 31 departures from the Determination, which were of an administrative nature. Such departures may help identify areas in the current processes where greater clarity is needed or where training or education for members is needed. These departures were as follows:

• 12 claims were not submitted for payment within 60 days of receipt or occurrence of the expense
• 10 annual loyalty scheme declarations were submitted by members after the due date specified in the guideline
• 7 reconciliations for the Sydney Allowance were submitted after the due date
• 2 publications claimed under the Communications Allowance had not made the required authorisations and attributions on the publication.

Background

The Parliamentary Remuneration Tribunal (the Tribunal) determines the salary and additional entitlements of members of the NSW Parliament (members), details of which are set out in the Tribunal's annual Determination. The NSW Parliament, through the Department of Parliamentary Services (the department), administers payments of additional entitlements to members in accordance with the Tribunal's annual Determination. An overview is presented below:

Twelve claims were not submitted for payment within 60 days of receipt or occurrence of the expense

The Determination requires members' expense claims to be submitted to the department within 60 days of when the expense is incurred or receipted. Our audit procedures identified 12 instances where members submitted their claims between six and 248 days late.

Ten annual loyalty/incentive scheme declarations were submitted by members after the due date specified in the guidelines

At the end of each financial year, members must declare they have not used loyalty/incentive scheme benefits accrued from their parliamentary duties for private purposes. The Determination requires current members to complete the declarations at the end of each year (by 27 August 2021 per the department's administrative process). Former members must complete the declarations within 30 days of leaving Parliament. We found ten current members submitted their declarations between three and 18 days late. The declaration is important as it affirms that loyalty benefits accrued using the members' parliamentary allowances and entitlements were not used for private purposes.

Seven reconciliations for the Sydney Allowance reconciliations were submitted after the due date

Open prior period recommendations

Enhanced public reporting

In 2016, the Auditor-General's Report to Parliament recommended the Tribunal consider requiring the department to regularly publish full details of members' expenditure claims on its website in an accessible and searchable format. The Tribunal had developed a plan requiring greater public reporting of members' additional expenditure from 1 July 2019 but does not have the power to require the department to facilitate this.

The Annual Reports of the Legislative Assembly and the Legislative Council, published on the Parliament's website, currently list the total amount claimed during the year by each member for each allowance. However, transparency around members’ claims would be enhanced if information was more extensively and regularly published on the Parliament’s website. The department should continue to work with the Presiding Officers, members, the Clerk of the Parliaments and the Clerk of the Legislative Assembly to enhance reporting of members' expenditure.

Clarifying key parameters of the annual Determination

In 2020, the Auditor-General's Reports to Parliament recommended the department work with the Tribunal to provide additional guidance to members to clarify:

• the definition of 'parliamentary duties'
• the activities that meet the definition
• requirements for retaining documents.

To address this recommendation, the department has performed a review of the definitions and activities used by other jurisdictions, in their administration of members' entitlements. The department is also continuing to monitor for changes in the administration of members' entitlements occurring at the Federal level. The department will work with the Tribunal to clarify these items as part of its submission to the 2022 annual Determination.

Resolved prior period recommendations

Recommendations resolved since the 2020 Auditor-General's report

The 2019 Auditor-General's Report recommended the department work with the Tribunal to clarify whether members can claim the cost of travel from their General Travel Allowance when the travel was used to produce communications during the blackout period. Members are not permitted to use their Communications Allowance for the production and distribution of publications that they intended to distribute in a State Election year in the period from 26 January to the election date (the ‘blackout period’).

The 2021 Determination has clarified this matter by stating that during the 'blackout period' travel necessary for parliamentary duties rather than electioneering is acceptable. The 2021 Determination has also included the condition that a member may not use their General Travel Allowance to fund communications that would normally be funded from the Communications Allowance during a 'blackout period'.

Appendix one - Response from Department of Parliamentary Services

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

This report analyses the results of our audits of the Premier and Cabinet cluster agencies for the year ended 30 June 2021.

Our preferred approach is to table the ‘Report on State Finances’ in Parliament before any other cluster report. This is because the 'Report on State Finances' focuses on the audit results and observations relating to the Total State Sector Accounts, in effect a consolidation of all government agencies. This year the 'Report on State Finances' has been delayed due to significant accounting issues being considered in the Total State Sector Accounts and which may impact the Treasury and Transport clusters.

As there are no outstanding matters relating to audits in the Premier and Cabinet cluster impacting the Total State Sector Accounts we have decided to break with normal practice and table this cluster report ahead of the ‘Report on State Finances’.

What the report is about

The results of the Premier and Cabinet cluster (the cluster) agencies' financial statement audits for the year ended 30 June 2021.

What we found

Unmodified audit opinions were issued for all Premier and Cabinet cluster agencies.

The number of monetary misstatements decreased from 49 in 2019–20 to 38 in 2020–21.

The Library Council of New South Wales corrected a prior period error of $325 million. In 2017, the council split its collection assets into six asset classes, but not the related asset revaluation reserves. To correct this error, some revaluation decrements previously recognised in asset revaluation reserves were reclassified to accumulated funds.

Eight agencies did not complete all of the mandatory early close procedures.

What the key issues were

The Premier and Cabinet cluster was impacted by three Machinery of Government (MoG) changes during 2020–21.

The changes resulted in the transfer of activities and functions in and out of the cluster and the creation of a new entity - Investment NSW.

The transferor entities continued to provide services to Investment NSW subsequent to 30 June 2021. There were no formal service level agreements in place for the provision of these services.

The New South Wales Electoral Commission (the Commission) and Sydney Opera House Trust obtained letters of financial support from their relevant Minister and/or NSW Treasury in 2020–21. The postponement of local government elections impacted the Commission's operations due to increased planned expenditure to support a COVID-safe election. Sydney Opera House Trust's ability to generate revenue was impacted due to the closure of the Concert Hall partly due to COVID-19 and planned renovations.

The number of repeated audit issues raised with management and those charged with governance increased from 22 in 2019–20 to 24 in 2020–21.

There were 47 moderate risk and 28 low risk findings identified. Of the total findings there were 24 repeat issues.

What we recommended

Investment NSW should ensure services received from other agencies are governed by service level agreements.

This report provides Parliament and other users of the Premier and Cabinet’s financial statements with the results of our audits, our observations, analysis, conclusions and recommendations in the following areas:

• financial reporting
• audit observations.

Financial reporting is an important element of good governance. Confidence and transparency in public sector decision making are enhanced when financial reporting is accurate and timely.

This chapter outlines our audit observations related to the financial reporting of agencies in the Premier and Cabinet cluster (the cluster) for 2021.

Appropriate financial controls help ensure the efficient and effective use of resources and administration of agency policies. They are essential for quality and timely decision making.

This chapter outlines our observations and insights from our financial statement audits of agencies in the Premier and Cabinet cluster.

Appendix one – Misstatements in financial statements submitted for audit

Appendix two – Early close procedures

Appendix three – Timeliness of financial reporting

Appendix four – Financial data

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

What the report is about

This audit assessed nine agencies’ compliance with the NSW Cyber Security Policy (CSP) including whether, during the year to 30 June 2020, the participating agencies:

• met their reporting obligations under the CSP
• reported accurate self-assessments of their level of maturity implementing the CSP’s requirements including the Australian Cyber Security Centre’s (ACSC) Essential 8.

What we found

Key elements to strengthen cyber security governance, controls and culture are not sufficiently robust and not consistently applied. The CSP is not achieving the objectives of improved cyber governance, controls and culture because:

• the CSP does not specify a minimum level for agencies to achieve in implementing the 'mandatory requirements' or the Essential 8
• the CSP does not require agencies to report their target levels, nor does it require risk acceptance decisions to be documented or formally endorsed
• each participating agency had implemented one or more of the mandatory requirements in an ad hoc or inconsistent basis
• none of the participating agencies had implemented all of the Essential 8 controls
• agencies tended to over-assess their cyber security maturity - all nine participating agencies were unable to support all of their self-assessments with evidence
• there is no monitoring of the adequacy or accuracy of agencies' self-assessments.

What we recommended

In this report, we repeat recommendations made in the 2019 and 2020 Central Agencies reports, that Cyber Security NSW and NSW Government agencies need to prioritise improvements to cyber security resilience as a matter of urgency.

Cyber Security NSW should:

• monitor and report compliance with the CSP
• require agencies to report the target and achieved levels of maturity
• require agencies to justify why it is appropriate to target a low level of maturity
• require the agency head to formally accept the residual risk
• challenge agencies' target maturity levels.

Agencies should resolve discrepancies between their reported level of maturity and the level they are able to support with evidence.

Separately, the agencies we audited requested that we not disclose our audit findings. We reluctantly agreed to anonymise our findings, even though they are more than 12 months old. We are of the view that transparency and accountability to the Parliament of New South Wales are part of the solution, not the problem.

The poor levels of agency cyber security maturity are a significant concern. Improvement requires leadership and resourcing.

This report assesses whether state government agencies are complying with the NSW Cyber Security Policy. The audit was based on the level of compliance reported at 30 June 2020.

Our audit identified non-compliance and significant weaknesses against the government’s policy.

Audited agencies have requested that we not report the findings of this audit to the Parliament of New South Wales, even though the findings are more than 12 months old, believing that the audit report would expose their weaknesses to threat actors.

I have reluctantly agreed to modify my report to anonymise agencies and their specific failings because the vulnerabilities identified have not yet been remedied. Time, leadership and prioritised action should have been sufficient for agencies to improve their cyber safeguards. I am of the view that transparency and accountability to the Parliament is part of the solution, not the problem.

The poor levels of cyber security maturity are a significant concern. Improvement requires dedicated leadership and resourcing. To comply with some elements of the government’s policy agencies will have to invest in technical uplift and some measures may take time to implement. However, other elements of the policy do not require any investment in technology. They simply require leadership and management commitment to improve cyber literacy and culture. And they require accountability and transparency. Transparent reporting of performance is a key means to improve performance.

Cyber security is increasingly a focus of governments around Australia. The Australian Cyber Security Centre (ACSC) is the Australian Government’s lead agency for cyber security and is part of the Australian Signals Directorate, a statutory authority within the Australian Government’s Defence portfolio. The ACSC has advised that government agencies at all levels, as well as individuals and other organisations were increasingly targeted over the 2021 financial year¹. The ACSC received over 67,500 cybercrime reports, a 13 per cent increase on the previous year. This equates to one reported cyber attack every eight minutes. They also noted that attacks by cyber criminals and state actors are becoming increasingly sophisticated and complex and that the attacks are increasingly likely to be categorised as ‘substantial’ in impact.

High profile attacks in Australia and overseas have included a sustained malware campaign targeted at the health sector², a phishing campaign deploying emotet malware, spear phishing campaigns targeting people with administrator or other high-level access, and denial of service attacks. The continuing trend towards digital delivery of government services has increased the vulnerability of organisations to cyber threats.

The COVID-19 pandemic has increased these risks. It has increased Australian dependence on the internet – to work remotely, to access services and information, and to communicate and continue our daily lives. Traditional security policies within an organisation’s perimeter are harder to enforce in networks made up of home and other private networks, and assets the organisation does not manage. This has increased the cyber risks for NSW Government agencies.

In March 2020, Service NSW suffered two cyber security incidents in short succession. Technical analysis undertaken by the Department of Customer Service (DCS) concluded that these cyber breaches resulted from a phishing exercise through which external threat actors gained access to the email accounts of 47 staff members. These attacks resulted in the breach of a large amount of personal customer information contained in these email accounts. These attacks were the subject of the Auditor-General's report on Service NSW's handling of personal information tabled on 18 December 2020.

This audit also follows two significant performance audits. Managing cyber risks, tabled on 13 July 2021 found Transport for NSW and Sydney Trains were not effectively managing their cyber security risks. Integrity of data in the Births, Deaths and Marriages Register, tabled 7 April 2020 found that although there are controls in place to prevent and detect unauthorised access to, and activity in the register, there were significant gaps in these controls.

The NSW Cyber Security Policy (CSP) was issued by Cyber Security NSW, a business unit within the Department of Customer Service, and took effect from 1 February 2019. It applies to all NSW Government departments and public service agencies, including statutory authorities. Of the 104 agencies in the NSW public sector that self-assessed their maturity implementing the mandatory requirements, only five assessed their maturity at level three or above (on the five point maturity scale). This means that, according to their own self-assessments, 99 agencies practiced requirements within the framework in what the CSP’s maturity model describes as an ad hoc manner, or they did not practice the requirement at all. Cyber Security NSW and NSW Government agencies need to prioritise improvements to their cybersecurity and resilience as a matter of priority.

This audit looks specifically at the compliance of nine key agencies with the CSP. It looks at their achievement implementing the requirements of the policy, the accuracy of their self-assessments and the attestations they made as to their compliance with the CSP.

The CSP outlines the mandatory requirements to which all NSW Government departments and public service agencies must adhere. It seeks to ensure cyber security risks to agencies’ information and systems are appropriately managed. The key areas of responsibility for agencies are:

• Lead - Agencies must implement cyber security planning and governance and report against the requirements outlined in the CSP and other cyber security measures.
• Prepare - Agencies must build and support a cyber security culture across their agency and NSW Government more broadly.
• Prevent - Agencies must manage cyber security risks to safeguard and secure their information and systems.
• Detect/Respond/Recover - Agencies must improve their resilience including their ability to rapidly detect cyber incidents and respond appropriately.
• Report - Agencies must report against the requirements outlined in the CSP and other cyber security measures.

DCS has only recommended, but not mandated the CSP for state owned corporations, local councils and universities.

NSW Government agencies must include an attestation on cyber security in their annual report and provide a copy to Cyber Security NSW by 31 August each year stating whether, for the preceding financial year, the agency has:

• assessed its cyber security risks
• appropriately addressed cyber security at agency governance forums
• a cyber incident response plan that is integrated with the security components of business continuity arrangements, and the response plan has been tested during the previous 12 months (involving senior business executives)
• certified the agency’s Information Security Management System (ISMS) or confirmed the agency’s Cyber Security Framework (CSF)
• a plan to continuously improve the management of cyber security governance and resilience.

The purpose of the attestation is to focus the agency's attention on its cyber risks and the mitigation of those risks.

Agencies assess their level of compliance in accordance with a maturity model. The CSP does not mandate a minimum maturity threshold for any requirement, including implementation of the Australian Cyber Security Centre's (ACSC) Essential 8 Strategies to Mitigate Cyber Security Incidents (Essential 8).

Agencies are required to set a target maturity level based on their risk appetite for each requirement, seek continual improvement in their maturity, and annually assess their maturity on an ascending scale of one to five for all requirements (refer to Appendix two for the maturity model). Each control within the Essential 8 is assessed on an ascending scale of zero to three reflecting the agency's level of alignment with the strategy (refer to Appendix three for the maturity model).

Scope of this audit

We assessed whether agencies had provided accurate reporting on their level of maturity implementing the requirements of the CSP in a documented way and covering all their systems.

The scope of this audit covered nine agencies (the participating agencies). These agencies were selected because they are the lead agency in their cluster, or have a significant digital presence within their respective cluster. The list of participating agencies is in section 1.2. The audit aimed to determine whether, during the year to 30th June 2020, the participating agencies:

• met their reporting obligations under the CSP
• provided accurate reporting in self-assessments against the CSP’s mandatory requirements, including their implementation of the Australian Cyber Security Centre’s (ACSC) Essential 8
• achieved implementation of mandatory requirements at maturity levels which meet or exceed the ‘level three - defined’ threshold (i.e. are documented and practiced on a regular and consistent basis).

While the audit does assess the accuracy of agency self-assessed ratings, the audit did not assess the appropriateness of the maturity ratings.

1. Key findings

The CSP allows agencies to determine their own level of maturity to implement the 'mandatory requirements', which can include not practicing a policy requirement or implementing a policy requirement on an ad hoc basis. These determinations do not need to be justified

Agencies can decide not to implement requirements of the CSP, or they can decide to implement them only in an informal or ad-hoc manner. The CSP allows agencies to determine their desired level of maturity in implementing the requirements on a scale of one to five - level one being 'initial – not practiced' and level five being 'optimised'. The desired level of maturity is determined by the agency based on their own assessment of the risk of the services they provide and the information they hold.

The reporting template for the 2019 version of the CSP stated that level three maturity - where a policy requirement is practiced on a regular and consistent basis and its processes are documented - was required for compliance with the CSP. This requirement was removed in the 2020 revision of the reporting template.

This CSP does not require the decisions on risk tolerance, or the timeframes agencies have set to implement requirements to be documented or formally endorsed by the agency head. There is no requirement to report these decisions to Cyber Security NSW.

Some comparable jurisdictions require formal risk acceptance decisions where requirements are not implemented. The NSW CSP does not have a similar formal requirement

Some jurisdictions, with a similar policy framework to NSW, require agencies to demonstrate reasons for not implementing requirements, and require agency heads to formally acknowledge the residual risk. The NSW CSP does not require these considerations to be documented, nor does it require an explicit acknowledgement and acceptance of the residual risk by the agency head or Cyber Security NSW. The NSW CSP does not require that the records of how agencies considered and decided which measures to adopt to be documented and auditable, limiting transparency and accountability of decisions made.

All of the participating agencies had implemented one or more of the mandatory requirements in an ad hoc or inconsistent basis

All of the participating agencies had implemented one or more of the mandatory requirements at level one or two. Maturity below level three typically means not all elements of the requirement have been implemented, or the requirements have been implemented on an ad-hoc or inconsistent basis.

None of the participating agencies has implemented all of the Essential 8 controls at level one – that is, only partly aligned with the intent of the mitigation strategy

Eight of the nine agencies we audited had not implemented any of the Essential 8 strategies to level three – that is, fully aligned with the intent of the mitigation strategy. At the time of this audit the ACSC advised that:

as a baseline organisations should aim to reach to reach Maturity Level Three for each mitigation strategy³.

The Australian Signals Directorate⁴ currently advises that, with respect to the Essential 8:

[even] level three maturity will not stop adversaries willing and able to invest enough time, money and effort to compromise a target. As such, organisations still need to consider the remainder of the mitigation strategies from the Strategies to Mitigate Cyber Security Incidents and the Australian Government Information Security Manual

All agencies failed to reach even level one maturity for at least three of the Essential 8.

Cyber Security NSW modified the ACSC model for implementation of the Essential 8

The NSW maturity model used for the Essential 8 does not fully align with the ACSC’s model. At the time of this audit the major difference was the inclusion of level zero in the NSW CSP maturity scale. Level zero broadly means that the relevant cyber mitigation strategy is not implemented or is not applied consistently. Level zero had been removed by the ACSC in February 2019 and was not part of the framework at the time of this audit. It was re-introduced in July 2021 when the ACSC revised the detailed criteria for each element of the essential 8 maturity model. The indicators to reach level one on the new ACSC model are more detailed, specific and rigorous than those currently prescribed for NSW Government agencies. Cyber Security NSW asserted the level zero on the CSP maturity scale:

is not identical to the level zero of the ACSC’s previous Essential 8 maturity model, but is a NSW-specific inclusion designed to prevent agencies incorrectly assessing as level one when they have not achieved that level.

Attestations did not accurately reflect whether agencies implemented the requirements

Of the nine participating agencies, seven did not modify the proforma wording in their attestation to reflect their actual situation. Despite known gaps in their implementation of mandatory requirements, these agencies stated that they had 'managed cyber security risks in a manner consistent with the Mandatory Requirements set out in the NSW Government Cyber Security Policy'. Only two agencies modified the wording of the attestation to reflect their actual situation.

Attestations should be accurate so that agencies’ and the government’s response to the risk of cyber attack is properly informed by an understanding of the gaps in agency implementation of the policy requirements and the Essential 8. Without accurate information about these gaps, subsequent decisions as to prioritisation of effort and deployment of resources are unlikely to effectively mitigate the risks faced by NSW Government agencies.

Participating agencies were not able to support all of their self-assessments with evidence and had overstated their maturity assessments, limiting the effectiveness of agency risk management approaches

Seven of the nine participating agencies reported levels of maturity against both the mandatory requirements and the Essential 8 that were not supported by evidence.

Each of the nine participating agencies for this audit had overstated their level of maturity against at least one of the 20 mandatory requirements. Seven agencies were not able to provide evidence to support their self-assessed ratings for the Essential 8 controls.

Where agency staff over-assess the current state of their cyber resilience, it can undermine the effectiveness of subsequent decision making by Agency Heads and those charged with governance. It means that actions taken in mitigating cyber risks are less likely to be appropriate and that gaps in implementing cyber security measures will remain, exposing them to cyber attack.

Agencies' self-assessments across government exposed poor levels of maturity in implementing the mandatory requirements and the Essential 8 controls

We reviewed the data 104 NSW agencies provided to Cyber Security NSW. The 104 agencies includes nine audited agencies referred to in more detail in this report. Our review of the 104 agency self-assessment returns submitted to Cyber Security NSW highlighted that, consistent with previous years, there remains reported poor levels of cyber security maturity. We reported the previous years’ self-assessments in the Central Agencies 2019 Report to Parliament and the Central Agencies 2020 Report to Parliament.

Only five out of the 104 agencies self-assessed that they had implemented all of the mandatory requirements at level three or above (against the five point scale). Fourteen agencies self-assessed that they had implemented each of the Essential 8 controls at level one maturity or higher (using Cyber NSW’s four point scale). The remainder reported at level zero for implementation of one or more of the Essential 8 controls, meaning that for the majority of agencies the cyber mitigation strategy has not been implemented, or is applied inconsistently.

Where agencies had reported in both 2019 and 2020, agencies’ self-assessments showed little improvement over the previous year’s self-assessments:

• 14 agencies reported improvement across both the Essential 8 and the mandatory requirements
• 8 agencies reported a net decline in both the Essential 8 and the mandatory requirements.

The poor levels of maturity in implementing the Essential 8 over the last couple of years is an area of significant concern that requires better leadership and resourcing to prioritise the required significant improvement in agency cyber security measures.

2. Recommendations

Cyber Security NSW should:

1. monitor and report compliance with the CSP by:

• obtaining objective assurance over the accuracy of self-assessments
• requiring agencies to resolve inaccurate or anomalous self-assessments where these are apparent

2. require agencies to report:

• the target level of maturity for each mandatory requirement they have determined appropriate for their agency
• the agency head's acceptance of the residual risk where the target levels are low

3. identify and challenge discrepancies between agencies' target maturity levels and the risks of the information they hold and services they provide

4. more closely align their policy with the most current version of the ACSC model.

Participating agencies should:

5. resolve the discrepancies between their reported level of maturity and the level they are able to demonstrate with evidence, and:

• compile and retain in accessible form the artefacts that demonstrate the basis of their self-assessments
• refer to the CSP guidance when determining their current level of maturity
• ensure the attestations they make refer to departures from the CSP
• have processes whereby the agency head and those charged with governance formally accept the residual cyber risks.

Repeat recommendation from the 2019 Central Agencies report and the 2020 Central Agencies report

6. Cyber Security NSW and NSW Government agencies need to prioritise improvements to their cyber security and resilience as a matter of urgency.

The objective of the CSP is to ensure cyber security risks are appropriately managed. However, meeting this objective depends on the requirements being implemented at all agencies to a level of maturity that addresses their specific cyber security risks. Agency systems and data are increasingly interconnected. If an agency does not implement the requirements, or implements them only in an ad-hoc or informal way, an agency is more susceptible to their systems and data being compromised, which may affect the confidentiality of citizens' data and the reliability of services, including critical infrastructure services.

Agencies determine their own target level of maturity, which may mean the requirement is not addressed, or is addressed in an ad hoc or inconsistent way

While the CSP is mandatory for all agencies, it does not set a minimum maturity threshold for agencies to meet.

The reporting template issued in 2019 stated that agencies were required to reach level three maturity in order to comply with the CSP. The 2020 revision⁶ of the CSP and guidance indicates that level three maturity may not be sufficient to mitigate risks. It advises the agency may determine the level to which it believes it is suitable to implement the requirements, and allows for an agency to aim for a target level of maturity less than level three. The agency can set its optimal maturity level with reference to its risk tolerance with the objective that that aim ‘to be as high as possible’. However, ‘as high as possible’ does not necessarily mean ‘fully implemented’. The CSP contemplates that a lower level of maturity is sufficient if it aligns with the agency's risk tolerance.

The Department of Customer Service asserts that while the quotes above were part of their annual templates and policy documents, their documents were incorrect. They assert that the policy has never required a minimum level of maturity to be reached. They have responded to our enquiries that:

…a level three maturity was not a requirement of the Policy or Maturity Model’ and ‘it is misleading to suggest it was a requirement of the Policy.

This audit found that, based on the 2020 reporting template there is no established minimum baseline. Consequently, because the Department of Customer Service had not established a minimum baseline agencies are able to target lower levels (providing they were within the agency’s own risk appetite), which includes targeting to not practice a CSP policy requirement, or to practice a CSP policy requirement on an ad hoc basis.

Where requirements are not implemented, documentation of formal acceptance of the residual risks by the agency head is not required

The New Zealand Government has an approach that is not dissimilar to NSW, in that it also identifies 20 mandatory requirements and allows for a risk based approach to implementation. However, the New Zealand approach puts more rigor around risk acceptance decisions.

The New Zealand Government requires that agencies that do not implement the requirements must demonstrate that a measure is not relevant for them. It requires agencies to document the rationale for not implementing the measure, including explicit acknowledgement of the residual risk by the agency head. They require these records to be auditable.

A security measure with a ‘must’ or ‘must not’ compliance requirement is mandatory. You must implement or follow mandatory security measures unless you can demonstrate that a measure is not relevant in your context.

Not using a security measure without due consideration may increase residual risk for your organisation. This residual risk needs to be agreed and acknowledged by your organisation head.

A formal auditable record of how you considered and decided which measures to adopt is required as part of the governance and assurance processes within your organisation.

The NSW CSP does not require these considerations to be documented or auditable and does not require an explicit acknowledgement or acceptance of the residual risk by the agency head.

None of the participating agencies achieved level three implementation for all mandatory risk prevention and mitigation requirements

Maturity level three is the minimum level whereby an agency has implemented documented processes that are practiced on a regular basis across their environment. An agency has not reached level three if the requirement is implemented on an ad-hoc or inconsistent basis, or if not all elements of the requirement have been implemented.

None of the participating agencies achieved level three implementation for all mandatory requirements.

The requirements of the CSP are organised into five sections. Agency implementation of these requirements is discussed in the next five sections of this report.

• Lead: Planning and governance requirements. Section 2.1
• Prepare: Cyber security culture requirements. Section 2.2
• Prevent: Managing cyber incident prevention requirements. Section 2.3
• Detect/Respond/Recover: Resilience requirements. Section 2.4
• Report: Reporting requirements. Section 2.5.

Appendix one – Response from agencies

Appendix two – The maturity model for the mandatory requirements

Appendix three – Essential 8 maturity model

Appendix four – About the audit

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

What the report is about

This audit assessed how effectively Transport for NSW (TfNSW) and Sydney Trains identify and manage their cyber security risks.

The NSW Cyber Security Policy (CSP) sets out 25 mandatory requirements for agencies, including implementing the Australian Cyber Security Centre’s Essential 8 strategies to mitigate cyber security incidents, and identifying the agency’s most vital systems, their ‘crown jewels’. 

The audited agencies have requested that we do not disclose detail of the significant vulnerabilities detected during the audit, as these vulnerabilities are not yet remediated. We provided a detailed report to the agencies in December 2020 outlining significant issues identified in the audit. We have conceded to the agencies' request but it is disappointing that transparency to the Parliament and the public on issues that potentially directly affect them needs to be limited in this way.

What we found

TfNSW and Sydney Trains are not effectively managing their cyber security risks.

Both agencies have assessed their cyber security risks as unacceptably high and both agencies had not identified all of the risks we detected during this audit – some of which are significant.

Both agencies have cyber security plans in place that aim to address cyber security risks. TfNSW and Sydney Trains have combined this into the Transport Cyber Defence Rolling Program, part of the Cyber Defence Portfolio (CDP). 

However, neither agency has reached its target ratings for the CSP and the Essential 8 and maturity is low in relation to significant risks and vulnerabilities exposed.

Further, neither agency is fostering a culture where cyber security risk management is an important and valued aspect of decision-making.

TfNSW is not implementing cyber security training effectively across the cluster with only 7.2% of staff having completed basic cyber security training.

What we recommended

TfNSW and Sydney Trains should:

• develop and implement a plan to uplift the Essential 8 controls to the agency's target state
• as a matter of priority, address the vulnerabilities identified as part of this audit and previously described in a detailed Audit Office report provided to both agencies
• ensure cyber security risk reporting to executives and the Audit and Risk Committee
• collect supporting information for the CSP self assessments 
• classify all information and systems according to importance and integrate this with the crown jewels identification process
• require more rigorous analysis to re-prioritise CDP funding 
• increase uptake of cyber security training.

TfNSW should assess the appropriateness of its target rating for each of the CSP mandatory requirements.

Department of Customer Service should:

• clarify the requirement for the CSP reporting to apply to all systems
• require agencies to report the target level of maturity for each mandatory requirement.

Response to requests by audited agencies to remove information from this report

In preparing this audit report, I have considered how best to balance the need to support public accountability and transparency with the need to avoid revealing information that could pose additional risk to agencies’ systems. This has involved an assessment of the appropriate level of detail to include in the report about the cyber security vulnerabilities identified in this audit.

In making this assessment, the audit team consulted with Transport for NSW (TfNSW), Sydney Trains, and Cyber Security NSW to identify content which could potentially pose a threat to the agencies’ cyber security.

In December 2020, my office also provided TfNSW and Sydney Trains with a detailed report of many of the significant vulnerabilities identified in this audit, to enable the agencies to address the cyber security risks identified. The detailed report was produced as a result of a 'red team' exercise, which was conducted with both agencies' knowledge and consent. The scope of this exercise reflected the significant input provided by both agencies. More information on this exercise is at page 12 of this report.

TfNSW and Sydney Trains have advised that in the six months from December 2020 and at the time of tabling this audit report, they have not yet remediated all the vulnerabilities identified. As a result, they, along with Cyber Security NSW, have requested that we not disclose all information contained in this audit report to reduce the likelihood of an attack on their systems and resulting harm to the community. I have conceded to this request because the vulnerabilities identified have not yet been remediated and leave the agencies exposed to significant risk.

It should be stressed that the risks identified in the detailed report exist due to the continued presence of these previously identified vulnerabilities, rather than due to their potential publication. The audited agencies, alone, are accountable for remediating these vulnerabilities and addressing the risks they pose.

It is disappointing that transparency to the Parliament and the public on issues that potentially directly affect them needs to be limited in this way.

That said, the conclusions drawn in this report are significant in terms of risk and remain valid, and the recommendations should be acted upon with urgency.

Cyber security risk is an increasing area of concern for governments in Australia and around the world. In recent years, there have been a number of high-profile cyber security attacks on government entities in Australia, including in New South Wales. Malicious cyber activity in Australia is increasing in frequency, scale, and sophistication. The Audit Office of New South Wales is responding to these risks with a program of audits in this area, which aim to identify the effectiveness of particular agencies in managing cyber risks, as well as their compliance with relevant policy.

Cyber Security NSW, part of the Department of Customer Service (DCS) releases and manages the NSW Cyber Security Policy (CSP). The CSP sets out 25 mandatory requirements for agencies, including making it mandatory for agencies to implement the Australian Cyber Security Centre Essential 8 Strategies to Mitigate Cyber Security Incidents (the Essential 8). The Essential 8 are key controls which serve as a baseline set of protections which agencies can put in place to make it more difficult for adversaries to compromise a system. Agencies are required to self-assess their maturity against the CSP and the Essential 8, and report that assessment to Cyber Security NSW annually.

The CSP makes agencies responsible for identifying and managing their cyber security risks. The CSP sets out responsibilities and governance regarding risk identification, including making agencies responsible for identifying their 'crown jewels', the agency's most valuable and operationally vital systems. Once these risks are identified, agencies are responsible for developing a cyber security plan to mitigate those risks.

This audit focussed on two agencies: Transport for NSW (TfNSW) and Sydney Trains. TfNSW is the lead agency for the Transport cluster and provides a number of IT services to the entire cluster, including Sydney Trains. This audit focussed on the activities of TfNSW's Transport IT function, which is responsible for providing cyber security across the cluster, as well as directly overseeing four of TfNSW's crown jewels. Sydney Trains is one of the agencies in the Transport cluster. While it receives some services from TfNSW, it is also responsible for implementing its own IT controls, as well as controls to protect its Operational Technology (OT) environment. This OT environment includes systems which are necessary for the operation and safety of the train network.

To test the mitigations in place and the effectiveness of controls, this audit involved a 'red team' simulated exercise. A red team involves authorised attackers seeking to achieve certain objectives within the target's environment. The red team simulated a determined external cyber threat actor seeking to gain access to TfNSW's systems. The red team also sought to test the physical security of some Sydney Trains' sites relevant to the agency's cyber security. The red team exercise was conducted with the knowledge of TfNSW and Sydney Trains.

This audit included the Department of Customer Service as an auditee, as they have ownership of the CSP through Cyber Security NSW. This audit did not examine the management of cyber risk in the Department of Customer Service.

This audit assessed how effectively selected agencies identify and manage their cyber security risks. The audit assessed this with the following criteria:

• Are agencies effectively identifying and planning for their cyber security risks?
• Are agencies effectively managing their cyber security risks?

Following this in-depth portfolio assessment, the Auditor-General for NSW will also table a report on NSW agencies' compliance with the CSP in the first quarter of 2021–22.

Agencies have assessed their cyber risks as being above acceptable levels

An agency's risk tolerance is the amount of risk which the agency will accept or tolerate without developing further strategies to modify the level of risk. Risks that are within an agency's risk tolerance may not require further mitigation and may be deemed acceptable, while risks which are above the agency's risk tolerance likely require further mitigation before they become acceptable to the agency.

Both agencies have defined their risk tolerance and have identified risks which are above this level, indicating that they are unacceptable to the agency. TfNSW has defined 'very high' risks as generally intolerable and 'high' risks as undesirable. Its risk tolerance is 'medium'. Sydney Trains has four classifications of risk: A, B, C and D. A and B risks are deemed 'unacceptable' and 'undesirable' respectively, while C risks are considered 'tolerable'. This aligns with the TfNSW definition of a medium risk tolerance.

Transport IT reported five enterprise-level cyber security risks through its enterprise risk reporting tool in September 2020, all of which relate to cyber security or have causes relating to cyber security. These risks are in aggregate form, rather than relating to specific vulnerabilities. At the time of the audit, one of these risks was rated as very high and the other four rated as high. At this time, Transport IT had identified a further seven divisional-level risks which were above the agency’s risk tolerance.

Similarly, Sydney Trains has identified one main cyber security risk in its IT enterprise-level risk register and another with a potential cyber cause. Both of these IT risks are deemed to have a residual risk of ‘unacceptable’.

Similarly, two cyber-related OT risks have been determined to be above the agency's risk tolerance. One risk is rated as 'unacceptable'. Another risk, while not entirely cyber rated, is rated 'undesirable' and is deemed to have some causes which may stem from a cyber-attack.

Agencies have assessed their current cyber risk mitigations as requiring improvement

In addition to the risk ratings stated above, at the time of the audit neither agency believed that its controls were operating effectively. Transport IT had rated the control environments for its cyber security enterprise risks as 'requires improvement'. Mitigations were listed in the risk register for these risks but, in some cases, they were unlikely to reduce the risk to the target state or by the target date. For example, one risk had actions listed as 'under review' and no further treatment actions listed, but a due date of July 2021, while another risk was being treated by the CDP with a due date of July 2021. The CDP identified in May 2020 that while the average risk identified as part of that program will be reduced to a medium level by this date, ten high risks will still remain. Given the delays in the program, this number may be higher. As such, it seems unlikely that the enterprise risk will be reduced to below a 'high' level by July 2021.

Sydney Trains’ IT and OT risk registers cross-reference controls and mitigations against the causes and consequences. The IT cyber security risk identified in the register had causes with no mitigations designed for them. Further, some of these causes did not have future mitigations designed for them. This risk also had controls in place which are identified as partially effective. For the unacceptable OT risk noted above, while there was a control designed for each of the potential causes, Sydney Trains had identified all of the controls in place as either partially effective or ineffective. This indicates that Sydney Trains was not effectively mitigating the causes of its cyber risks and, even where it had designed controls or mitigations, these were not always implemented to fully mitigate the cause of the risk.

Additional information on gaps in cyber mitigations which were exposed in the course of this audit has been detailed to both agencies. The Foreword of this report provides information about why this detail is not included here.

Essential 8 maturity is low across TfNSW and Sydney Trains and little progress was made in 2020

CSP mandatory requirement 3.2 states that agencies must implement the ACSC Essential 8. Agencies must also rate themselves against each of the Essential 8 on a maturity scale from zero to three and report this to Cyber Security NSW. A full list of the Essential 8 can be found in Exhibit 1. Both agencies have a low level of maturity against the Essential 8 not just in comparison to the targets they have set, but also in relation to the risks and vulnerabilities exposed. Both agencies have set target maturity ratings for the Essential 8 but none of the Essential 8 ratings across either agency are currently implemented to this level. Having a low level of Essential 8 maturity exposes both agencies to significant risks and vulnerabilities. Little progress was made between the 2019 and 2020 attestation periods.

Transport IT has set a target rating of three across all of the Essential 8. Sydney Trains has set a target rating of three for its IT systems. Sydney Trains had an interim target of two for its OT systems in 2020 and advised that this has since increased to three. It should be noted that not all the Essential 8 are applicable to OT systems.

None of the Essential 8 ratings across either agency are currently implemented to the target levels. Given that the Essential 8 provide the controls which are most commonly able to deter cyber-attacks, having maturity at a low level potentially exposes agencies to a cyber security attack.

Some work is underway across both TfNSW and Sydney Trains to improve the Essential 8 control ratings. The CDP provided some resources to the Essential 8 over 2019–20, with uplift focusing on specific systems. The CDP work in 2019 and 2020 relevant to the Essential 8 largely focussed on determining the current state of the Essential 8 and creating a target state roadmap. As a result, there was little improvement between the 2019 and 2020 attestation periods. The CDP has a workstream for the Essential 8 in its FY 2020–21 funding allocation, however as noted above in Exhibit 6 this was delayed as resources were redeployed to Project La Brea. Regardless, work on some specific aspects of the Essential 8 remain part of the 2020–21 CDP allocation, with workstreams allocated to improving three of the Essential 8. In addition, some work from Project La Brea should lead to an improvement in the Essential 8.

Sydney Trains' Cyber Uplift Program included a workstream which had in scope the uplift in the Essential 8 in IT. There were also other workstreams which aimed to improve some of the Essential 8 for OT systems. Work is also ongoing as part of the CDP to uplift these scores in Sydney Trains.

TfNSW and Sydney Trains have not reached their target maturity across the CSP mandatory requirements and TfNSW has not evaluated its cluster-wide target to ensure it is appropriate

Cyber Security NSW allows each agency to determine its target level of maturity for the first 20 CSP mandatory requirements. Agencies can tailor their target levels to their risk profile. Not reaching the target rating of the CSP mandatory requirements risks information and systems being managed inconsistently or not in alignment with good governance principles.

Sydney Trains has set its target level of maturity for IT and OT. All of Sydney Trains' target maturity levels are at least a three (defined), with a target of four (quantitatively managed) for many of the mandatory requirements. While Cyber Security NSW does not currently mandate a minimum level of maturity, in 2019 there was a requirement for each agency to target a minimum level of three.

Sydney Trains has not met its target ratings across the mandatory requirements.

The Transport Cyber Defence Rolling Program has a program KPI to ensure that the entire cluster reaches a minimum maturity level of three against all the CSP requirements by 2023. TfNSW has not reviewed its CSP mandatory requirement targets to determine if a three is desirable for all requirements or if a higher target level may be more appropriate. It is important for senior management to set cyber security objectives as a demonstration of leadership and a commitment to cyber security.

TfNSW has not met its target ratings across the mandatory requirements for its Group IT ISMS, which was the focus of this audit.

Both agencies claimed progress in their implementation of the mandatory requirements between 2019 and 2020. The audit did not seek to verify the self-assessed results from either agency.

Both agencies operate ISMS in line with the CSP

CSP mandatory requirement 3.1 requires agencies to implement an Information Security Management System (ISMS) or Cyber Security Framework (CSF), with scope at least covering systems identified as the agency's ‘crown jewels’. The ISMS or CSF should be compliant with, or modelled on, one or more recognised IT or OT standard. As noted in the introduction, an ISMS ‘consists of the policies, procedures, guidelines, and associated resources and activities, collectively managed by an organisation, in the pursuit of protecting its information assets.’ Both agencies operate an ISMS compliant with the CSP requirement.

As noted in the introduction, TfNSW operates four ISMS. The Transport IT ISMS is certified against ISO27001, the most common standard for ISMS certification. Three of TfNSW’s six crown jewels are managed within this ISMS. The other ISMS are not certified to relevant standards, though TfNSW claims that they align with relevant controls. This is sufficient for the purposes of the CSP.

Sydney Trains operates two ISMS, one for IT and another for OT. Neither of these are certified to relevant ISMS Standards, however there have been conformance reviews of both IT and OT with relevant standards. These ISMS cover all crown jewels in the agency.

There are currently 11 ISMS in operation across the Transport cluster. TfNSW has proposed moving towards a holistic approach to these ISMS, with the CDP Board responsible for governing the available security controls and directing agency IT and OT teams to implement these.

Agencies are not routinely conducting audits of third-party suppliers to ensure compliance with contractual obligations

CSP mandatory requirement 1.5 makes agencies accountable for the cyber risks of their ICT service providers and ensuring that providers comply with the CSP and any other relevant agency security policies. The ACSC has provided advice on what organisations should do when managing third party suppliers of ICT. The ACSC advises that organisations should use contracts to define cyber security expectations and seek assurance to ensure that these contract expectations are being met. While both agencies usually include specific cyber security expectations in contracts, neither is routinely seeking assurance that these expectations are being met.

The NSW Government has mandated the use of the 'Core& One' contract template for low-value IT procurements and the Procure IT contract template for high-value IT procurements. Both of these contracts contain space for the procuring agency to include cyber security controls for the contractor to implement. The Procure IT contract template also includes a right-to-audit clause which allows agencies to receive assurance around the implementation of these controls. TfNSW and Sydney Trains used the mandated contracts for relevant contracts examined as part of this audit.

TfNSW included security controls in all the contracts examined as part of this audit. Compliance with ISO27001 was the most commonly stated security expectation. Of the contracts examined as part of this audit, only one contract did not have a right-to-audit clause. This contract was signed in October 2016. While these clauses are in place, TfNSW rarely conducted these audits on its third-party providers. Of the eight TfNSW contracts examined in detail, only two of these had been audited to confirm compliance with the stated security controls.

Sydney Trains included security controls in all but one of the contracts examined as part of this audit. Sydney Trains did not require contractors to be compliant with ISO27001, but only required compliance with whole-of-government policies. Sydney Trains does not routinely conduct audits of its third-party suppliers, however it did conduct deep-dive risk analyses of its top ten highest risk IT suppliers. This involved a detailed review of both the suppliers' security posture and also the contract underpinning the relationship with the supplier.

The CDP funding for 2020–21 includes a workstream for strategic third-party contract remediation. This funding is to conduct some foundational work which will allow the CDP to make further improvements in future years. While this funding will not address gaps in contract requirements or management across all contracts, this workstream aims to reduce the risks posed by strategic suppliers covering critical assets. Similarly, work is currently underway as part of the CDP to conduct OT risk assessments for key suppliers to Sydney Trains in a similar way to the work undertaken for IT suppliers.

Sydney Trains has risk assessed its third-party suppliers but TfNSW has not done so

It is important to conduct a risk assessment of suppliers to identify high-risk contractors. This allows agencies to identify those contractors who may require additional controls stated in the contract, those who require additional oversight, and also where auditing resources are best targeted.

Sydney Trains has risk assessed all its IT suppliers and, as noted above, has conducted a deep-dive risk analysis of its top ten highest risk suppliers. TfNSW has not undertaken similar analysis of its key suppliers, however it has identified risks attached to each of its strategic suppliers and has documented these. As a result of not risk assessing its suppliers, TfNSW cannot take a targeted approach to its contract management.

TfNSW demonstrated poor records handling relating to the contracts examined as part of this audit

TfNSW was not able to locate one of the contracts requested as part of the audit's sample. Other documentation, such as contract management plans, could not be located for many of the other contracts requested as part of this audit. These poor document handling practices limits TfNSW's ability to effectively oversee service providers and ensure that they are implementing agreed controls. It also limits public transparency on the effectiveness of these controls.

The Transport cluster is not effectively implementing cyber security awareness training

Agencies are responsible for implementing regular cyber security education for all employees and contractors under mandatory requirement 2.1 in the CSP. TfNSW is responsible for delivering this training to the whole Transport cluster, including Sydney Trains. The Transport cluster has basic cyber awareness training available for all staff. TfNSW also offers additional training provided by Cyber Security NSW targeted at executives and executive assistants. While TfNSW has training available to staff, it is not delivering this effectively. TfNSW does not make training mandatory for most staff nor does it require staff to repeat training regularly. Even among those staff who have been assigned the training, completion rates are low, meaning that delivery is not effectively monitored. Cyber security training is important for building and supporting a cyber security culture.

TfNSW is responsible for creating and rolling out all forms of training to agencies within the Transport cluster. Both TfNSW and Sydney Trains have the same mandatory cyber awareness training that is automatically assigned to new starters. At the time of the audit, this training was not mandatory for ongoing staff. TfNSW does make additional cyber security training available to staff who can choose to undertake the training themselves, or can be assigned the training by their manager. All TfNSW cyber security training is delivered via online modules and it is the responsibility of managers to ensure that it is completed.

Cyber security training completion rates for both TfNSW and Sydney Trains are low. Only 13.5 per cent of staff across the Transport cluster had been assigned the Cyber Safety for New Starters training as of January 2021. Although this course is mandatory for new starters, only 53 per cent of staff assigned the Cyber Safety for New Starters training module had completed the course by January 2021. As a result, only 7.2 per cent of staff across the entire Transport cluster had completed this training at that time. In Sydney Trains, less than one per cent of staff had completed this training as at January 2021 and a further 7.6 per cent of staff have completed the 'Cyber Security: Beyond the Basics' training. These low completion rates indicate that TfNSW is not effectively rolling out cyber security training across the cluster.

In October 2020, the Department of Customer Service released 'DCS-2020-05 Cyber Security NSW Directive - Practice Requirement for NSW Government', which made annual cyber security training mandatory for all staff from 2021. In line with this requirement, TfNSW has advised that it will be gradually implementing mandatory annual training from July 2021 for all staff.

The Transport cluster undertakes activities to build a cyber-aware culture in accordance with the CSP, but awareness remains low

Increasing staff awareness of cyber security risks and maintaining a cyber secure culture are both mandatory requirements of the CSP. While TfNSW does undertake some activities to build a cyber aware culture, awareness of cyber security risks remains low. This can be demonstrated by the low training rates outlined above, and the 'Spot the Scammer' exercise, described in Exhibit 7. TfNSW is responsible for delivering these awareness raising activities across the cluster.

TfNSW frequently communicates with staff across the Transport cluster about various cyber security risks through multiple avenues. Both agencies use the intranet, emails and other awareness raising activities to highlight the importance for staff to be aware of the seriousness of cyber risks. Advice given on the intranet includes tips for spotting scammers on mobile phones, promoting the cluster-wide training courses, as well as various advice that staff could use when dealing with cyber risks in the workplace.

In addition to these awareness raising activities, TfNSW has also undertaken a cluster-wide phishing email exercise called 'Spot the Scammer'. This is outlined in Exhibit 7. This exercise was carried out in 2019 and 2020 and allowed the Transport cluster to measure the degree to which staff were able to identify phishing emails. As can be seen in Exhibit 7, the results of this exercise indicate that staff awareness of phishing emails remains low.

Appendix one – Response from agencies

Appendix two – Cyber Security Policy mandatory requirements

Appendix three – About the audit

Appendix four – Performance auditing

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #353 - released (13 July 2021).

What the report is about

Results of the financial statement audits of the public universities in NSW for the year ended 31 December 2020.

What we found

Unqualified audit opinions were issued for all ten universities.

Two universities reported retrospective corrections of prior period errors.

Universities were impacted by the COVID-19 pandemic with student enrolments decreasing in 2020 compared to 2019 by 10,032 (3.3 per cent). Of this decrease 8,310 students were from overseas.

In response to the pandemic, each university provided welfare support, created student hardship funds, provided accommodation and flexibility on payment of course fees. State and Commonwealth governments provided additional support to the sector.

Six universities recorded negative net operating results in 2020 (two in 2019). The combined revenues of the ten universities from fees and charges decreased by $361 million (5.8 per cent).

Despite the impact of the COVID-19 pandemic, which will continue to impact the financial results of universities in 2021, enrolments of overseas students in semester one of 2021 increased at two universities. This growth meant that total overseas student enrolments increased by 7,944 or 5.8 per cent across the sector as a whole. However, eight universities experienced decreases in overseas student enrolments compared to semester one of 2020. All universities have experienced growth in domestic student enrolments.

What the key issues were

There were 110 findings reported to universities in audit management letters.

Three high risk findings were identified. One related to the continued work by the University of New South Wales to assess its liability for underpayment of casual staff entitlements. The other two deficiencies were at Charles Sturt University, relating to financial reporting implications of major contracts, and resolving issues identified by an internal review of its employment contracts to reliably quantify the university’s liability to its employees.

What we recommended

Universities should prioritise actions to address repeat findings. Forty-five findings were repeated from 2019, of which 23 related to information technology.

Further information

Please contact Ian Goodwin, Deputy Auditor-General on 9275 7347 or by email.

This report analyses the results of our audits of the financial statements of the ten universities in NSW for the year ended 31 December 2020. The table below summarises our key observations.

1. Financial reporting

2. Internal controls and governance

3. Teaching and research

This report provides Parliament with the results of our financial audits of universities in NSW and their controlled entities in 2020, including our analysis, observations and recommendations in the following areas:

• financial reporting
• internal controls and governance
• teaching and research.

Financial reporting is an important element of governance. Confidence and transparency in university sector decision making are enhanced when financial reporting is accurate and timely.

This chapter outlines our audit observations on the financial reporting of universities in NSW for 2020.

Financial results

The graph below shows the net results of individual universities for 2020.

Appropriate and robust internal controls help reduce risks associated with managing finances, compliance and administration of universities.

This chapter outlines the internal controls related observations and insights across universities in NSW for 2020, including overall trends in findings, level of risk and implications.

Our audits do not review all aspects of internal controls and governance every year. The more significant issues and risks are included in this chapter. These along with the less significant matters are reported to universities for management to address.

Universities' primary objectives are teaching and research. They invest most of their resources to achieve quality outcomes in academia and student experience. Universities have committed to achieving certain government targets and compete to advance their reputation and their standing in international and Australian rankings.

This chapter outlines teaching and research outcomes for universities in NSW for 2020.

Appendix one – Status of 2019 recommendations

Appendix two – Universities' controlled entities

A report released today by the Auditor-General for New South Wales, Margaret Crawford found that select advertising campaigns conducted by Service NSW and the NSW Rural Fire Service met most requirements of the Government Advertising Act, regulations, Guidelines and other laws. However, the audit found that Service NSW inappropriately used its post campaign evaluation to measure sentiment towards and confidence in the NSW Government.  

While agency analysis shows that the ‘Cost of Living’ (phases 2 and 3)  and ‘How Fireproof is Your Plan?’ campaigns achieved most of their objectives, the campaign objectives and targets set by both agencies were not sufficient to measure all aspects of campaign effectiveness. 

The report makes two recommendations to the Department of Customer Service. The first is to review its guidance to ensure agencies are not using post campaign evaluations to measure sentiment towards the government. The second, to review its guidance and the new process of peer review to ensure they support agencies to comply with the Act, the regulations and the Guidelines. 

The Government Advertising Act 2011 requires the Auditor General to conduct an annual performance audit of one or more government agencies to see whether their advertising activities were carried out in an effective, economical and efficient manner and in compliance with the Government Advertising Act 2011.
 

Read full report (PDF)

The Government Advertising Act 2011 (the Act) requires the Auditor-General to conduct a performance audit on the activities of one or more government agencies in relation to government advertising campaigns in each financial year. The performance audit assesses whether a government agency or agencies have carried out activities in relation to government advertising in an effective, economical and efficient manner and in compliance with the Act, the regulations, other laws and the Government Advertising Guidelines (the Guidelines). This audit examined two campaigns run during the 2018–19 and 2019–20 financial years respectively:

• the 'Cost of Living' campaign run by Service NSW (phases 2 and 3 delivered in 2018–19)
• the 'How Fireproof Is Your Plan?' (Fireproof) campaign run by NSW Rural Fire Service (year two of a three-year campaign delivered in 2019–20).

Section 6 of the Act prohibits political advertising. Under this section, material that is part of a government advertising campaign must not contain the name, voice or image of a minister, member of parliament or a candidate nominated for election to parliament or the name, logo or any slogan of a political party. Further, a campaign must not be designed to influence (directly or indirectly) support for a political party.

In 2018–19, Service NSW delivered phases 2 and 3 of the 'Cost of Living' campaign. The Cost of Living advertising campaign aimed to build awareness of the help available to ease the cost of living for people under financial pressure including awareness of specific rebates that can be claimed. As part of the Cost of Living program, Service NSW developed a webpage designed as a single portal to access more than 40 NSW Government savings, rebates and initiatives (which originated from over 12 different agencies). It also launched the Cost of Living service which includes face to face meetings and phone interviews to help people claim rebates from the NSW Government. Phase 2 of the campaign ran from September 2018 to August 2019. Phase 3 of the campaign ran from January 2019 to July 2019. The budgets for phases 2 and 3 were $4.127 million and $934,800 respectively. See Appendix two for more details on this campaign.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign including radio transcripts, digital videos and display. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. See Appendix two for examples of campaign materials for this campaign.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

Post-campaign evaluations measured sentiment towards and confidence in the NSW Government

The post-campaign evaluation for phases 2 and 3 measured levels of confidence with the statement ‘the NSW Government has your best interests at heart’, despite the fact this was not a stated objective of the campaign. This is not an appropriate use of the post-campaign evaluation, which should measure the success of the campaign against its stated objectives. The post-campaign evaluation for phase 3 found that exposure to the campaign improved sentiment towards the government amongst those who did not have confidence in the NSW Government.

Service NSW advised that it was important to measure the sentiment of the advertising including the wording 'best interests' as it did not want the whole of government brand to be detrimental to customer engagement with applying for the rebates.

Following phase 2, Service NSW conducted analysis of media sentiment using the key words 'cost of living' and the names of the Premier, Treasurer and Minister for Customer Service. The analysis presented the level of positive, negative and neutral media sentiment. The Government Advertising Guidelines 2012 list the purposes that government advertising may serve which do not include improving the perception of the government. The inclusion of this analysis in Service NSW's post-campaign evaluation creates a risk that the results may be used for party political purposes.

Section 10 of the Act restricts agencies from carrying out a campaign after 26 January in the calendar year before the Legislative Assembly is due to expire and before the election for the Legislative Assembly in that year. Service NSW authorised a media agency to book media in line with the media plans for the campaign. The media plans for the campaign show that Service NSW did not authorise or plan to run any advertisements between 27 January 2019 and 23 March 2019.

Service NSW did not set targets for all rebates advertised in phase 2

Service NSW did not set targets for four of the seven rebates that were advertised as part of phase 2 of the campaign. These rebates were the Family Energy Rebate, Appliance Replacement Offer, National Parks Concession Offer and the Pensioner Travel Voucher. As a result, it was unable to evaluate whether the advertisements for these rebates were effective. Service NSW advised that at the time the campaign went to peer review, when campaign objectives are set, it did not know which rebates would be included in the advertisements.

Service NSW stated in its submission to the Department of Premier and Cabinet that it may change the creative content for phase 2 as it announced new initiatives and rebates. The peer review process should have ensured that Service NSW set targets for any additional rebates or savings it intended to advertise before that advertising commenced to ensure a strategic approach to the campaigns that clearly demonstrated anticipated benefits were in place.

The post-campaign evaluation for phase 2 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for some rebates advertised as part of phase 2 of the campaign. The objectives, targets and results for phase 2 are shown in Exhibit 5. In phase 2, Service NSW established baseline data on levels of awareness of government rebates during the peer review process. The baseline level of awareness for government rebates was 44 per cent. The level of awareness for specific rebates was 46 per cent for the Compulsory Third Party (CTP) green slip refund, and 21 per cent for both Active Kids and Toll Relief.

Post-campaign evaluation reports for phase 2 show that the campaign met its objective to raise awareness of NSW Government rebates, achieving a 16 per cent increase in awareness from 44 per cent to 51 per cent. The campaign did not meet its target to increase awareness of the CTP green slip refund by ten per cent.

Service NSW did not report the results of the uptake of the CTP green slip refund, Active Kids and Toll Relief in its post campaign effectiveness report submitted to the Department of Premier and Cabinet. However, other post-campaign evaluation documentation, which Service NSW advise was submitted to the Department of Premier and Cabinet, show that these targets were met.

Service NSW did not report to the Department of Premier and Cabinet on whether it achieved the target of a ten per cent increase of average monthly visits to the Cost of Living webpage. Service NSW reported that it had achieved an average of 11,753 visitors to the webpage per day during the campaign. These average daily results indicate that the target was met.

Exhibit 5: Phase 2 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. a) Increase awareness of rebates from the NSW Government by ten per cent.	Image
b) Increase average monthly visits to the Cost of Living webpage by ten per cent.	Image
2. Increase awareness of rebates and savings by ten per cent for:
CTP green slip refund	Image
Active Kids	Image
Toll Relief.	Image
3. Increase awareness that NSW Government initiatives relating to the cost of living are available via Service NSW by ten per cent.	Image
4. Increase the uptake of rebates and savings for the CTP green slip refund, Active Kids and Toll Relief by ten per cent.	Image

Key

Image

Yes

Image

Not Fully

The post-campaign evaluation for phase 3 shows that the advertising campaign met most of its objectives

Service NSW set overall campaign objectives and specific targets for the two rebates advertised as part of phase 3 of the campaign. The objectives, targets and results for phase 3 are shown in Exhibit 6.

In phase 3, Service NSW established baseline data on levels of awareness during the peer review process. The baseline level of awareness for government rebates was 44 per cent. This is the same baseline that was used to measure performance for phase 2 of the campaign. Service NSW did not set baselines for awareness and uptake of Energy Switch and Creative Kids as these were new services.

Post-campaign evaluation reports for phase 3 show that the campaign met its objective to raise awareness of NSW Government rebates by ten per cent, achieving a 30 per cent increase in awareness from 44 per cent to 57 per cent. The overall increase in message take-out was met with 43 per cent agreeing with the message that the NSW Government is taking steps to ease the cost of living. The campaign achieved awareness and uptake targets for the specific rebates included in phase 3, except for awareness of Creative Kids which achieved 28 per cent awareness, falling short of the 30 per cent awareness target.

Exhibit 6: Phase 3 - campaign objectives, targets and results

Campaign objectives and targets	Does the post-campaign evaluation show that the target was met?
1. Increase message takeout that ‘The NSW Government is taking steps to help ease the cost of living in NSW’ by ten per cent for those who can recall the campaign.
2. Increase awareness that the NSW Government has a range of rebates and savings by ten per cent.
3. Generate awareness with NSW residents aged 18+ of:
Energy Switch (15 per cent awareness)
Creative Kids (30 per cent awareness).
4. Create uptake of Energy Switch and Creative Kids (8,356 clicks on the Energy Switch website and 107,938 Creative Kids vouchers downloaded with 70 per cent conversion).

Key

Yes

Not Fully

The timing of campaign phases meant that it was difficult for Service NSW to evaluate each distinct campaign phase and reduced opportunities to incorporate learnings from previous phases

Service NSW commenced planning for phase 2 of the campaign while phase 1 was still underway. This limited the opportunity for Service NSW to incorporate learnings from phase 1 into phase 2. There was some overlap in the timing of phase 2 and the start of phase 3 of the campaign, making it difficult to evaluate the effectiveness of each distinct campaign phase. Both phases 2 and 3 had the same high-level outcome objective to raise awareness of rebates by ten per cent. The baseline measures that were used to evaluate performance for phase 3 were the same as those used to evaluate phase 2. As a result, Service NSW was not able to separately evaluate these two phases of the campaign. This is important given the budgets for phases 2 and 3 were $4.127million and $934,800 respectively.

Service NSW allocated 7.5 per cent of its media budget to communications with culturally and linguistically diverse (CALD) and Aboriginal audiences

The NSW Government CALD and Aboriginal Advertising Policy requires that agencies spend at least 7.5 per cent of an advertising campaign media budget on direct communications with CALD and Aboriginal audiences. Service NSW authorised a media company to book media in line with the media plans for the campaign. The media plans for phases 2 and 3 of the campaign indicate that Service NSW met this requirement, with 7.5 per cent of the budget allocated to these audiences in phase 2 and 10.4 per cent in phase 3.

The post campaign evaluation for phases 1 and 2 of the Cost of Living campaign contained a recommendation to look at other opportunities to reach CALD audiences. Effective communication with CALD audiences was particularly important in phase 3 of the campaign, where they made up 30 per cent of the target audience for the Creative Kids advertisement. The post-campaign analysis for phase 3 showed that the campaign performed well with some, but not all CALD audiences. The post-campaign analysis also showed low awareness and uptake with Aboriginal audiences. Pre-campaign focus groups in phase 3 found Aboriginal audiences had a negative reaction to the campaign tag line ‘NSW Government is helping with the cost of living’ however this tagline was still used in some advertisements in phase 3.

The cost-benefit analysis (CBA) for phase 2 did not accurately assess the benefits of the campaign and did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. The CBA conducted by Service NSW for phase 2 includes $8 million in benefits attributed to the advertisements for the Energy Switch tool and $6.9 million in benefits attributed to the advertisements for Creative Kids vouchers. These benefits should not have been included in the CBA for phase 2 as they were not included in this phase of the campaign. The CBA did not estimate the benefits of some other rebates and savings advertised in phase 2 of the campaign. This means that the CBA did not accurately assess the benefits of the campaign. Service NSW advised that at the time the CBA was developed it had not selected the rebates to be included in the campaign.

The Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits. The CBA developed as part of phase 2 identified using existing NSW Government communication channels as an alternative to advertising but did not assess the costs and benefits of this alternative.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Service NSW negotiated with a single creative agency in phase 2, making it difficult to demonstrate value for money

Agencies are required to obtain three quotes when procuring a creative agency on the prequalification scheme if the estimated cost of the creative content is greater than $150,000. In phase 2 of the campaign, Service NSW extended the contract with the creative agency used for phase 1 of the campaign and did not obtain three quotes despite the cost of the creative content for phase 2 being $731,480. The requirement to obtain three quotes was met in phase 1 when initially selecting this creative agency.

Service NSWs procurement policy details that direct negotiation may be appropriate where there is a compelling reason to renew or rollover a contract beyond temporal or convenience reasons or in the cases of a genuine emergency. In its briefing to the Chief Executive, Service NSW stated that this contract extension was sought due to the time-sensitive nature of the project and that if work was delayed by a tender process, Service NSW may not be able to meet marketing milestones and this could result in limited customer uptake. This reason is not a genuine emergency and is not compelling as it does not explain what consequences would occur if it did not meet the marketing milestones or if there was limited customer uptake.

Service NSW's procurement policy also states that under no circumstances must Service NSW employees enter into discussions with a supplier until the delegate has formally made their decision to enter into direct negotiation. Service NSW briefed the Chief Executive of Service NSW in relation to extending the contract on 5 September 2018. The briefing states that the creative agency had already begun developing creative content for phase 2 and Service NSW had already received quotes from the creative provider for the proposed work prior to 5 September 2018. Procurement sign-offs were not completed until 7 September 2018. The engagement of the creative provider prior to appropriate approvals was contrary to Service NSWs procurement policy.

The economy of the campaign may have been limited by not meeting the procurement requirements in phase 2. It is possible that the creative provider may have offered a more competitive rate if it was aware that Service NSW was seeking quotes from other creative providers. Additionally, it is possible that another creative provider could have provided better value for money.

In phase 3 of the campaign, the estimated cost of the creative exceeded $150,000 however Service NSW chose to contract two different creative agencies, and the cost for each agency fell below the threshold to obtain three quotes. Agencies are permitted to obtain one quote when using a creative provider on the prequalification scheme if the cost is between $50,000 to $150,000. Service NSW advised that it contracted two creative providers as two different project teams were responsible for the rebates, each with separate marketing budgets.

Service NSW allowed sufficient time for cost-efficient media placement

During the peer review process, the Department of Premier and Cabinet advised agencies about the time they should allow to ensure cost-efficient media placement. For example, the Department of Premier and Cabinet advised that agencies book television advertising six to 12 weeks in advance and that agencies book radio advertising two to eight weeks in advance.

Service NSW allowed sufficient time between the completion of the peer review process and the commencement of the first advertising. Service NSW signed the agreement with the approved Media Agency Services provider with sufficient time to achieve cost-efficient media placement for all types of media used in this campaign.

The campaign may have been misleading for some people who were not eligible for rebates

Advertisements we reviewed focused on the amount of savings that could be obtained from rebates, for example ‘Save up to $285’, and ended with a statement ‘To save, visit service.nsw.gov.au. This directed viewers to the Cost of Living website which contains eligibility information. However, the advertisements in phases 2 and 3 we reviewed did not contain any details on the eligibility for these rebates and not all advertisements stated that eligibility criteria apply. Service NSW advised that the eligibility criteria for each rebate is extensive and that it was not possible to include this in the creative material.

Post-campaign evaluations in phase 3 recommended that advertisements for Creative Kids should indicate eligibility (e.g. age criteria) as statements on savings have the potential to be misleading when not all viewers will be eligible for rebates. Social media analysis conducted following phase 2 showed ineligibility or inability to claim rebates or refunds caused anger for some respondents.

Some advertisements in phase 2 stated ‘we've got something for everyone’. However, as rebates were subject to eligibility criteria, it is possible that some residents in NSW would not be eligible for any rebates as part of the Cost of Living initiative. As such, this statement has the potential to be misleading.

The campaign included statements that underestimated the savings that some customers could obtain

The Guidelines require accuracy in the presentation of all facts, statistics, comparisons and other arguments. The Guidelines also require that all claims of fact included in government advertising campaigns must be able to be substantiated.

In phase 2, the possible savings customers could obtain for two rebates or savings exceeded the amounts stated in the advertising campaign. Exhibit 7 shows some advertisements in phase 2 which stated, ‘My Green Slip Saving Save up to $60’. However, the State Insurance Regulatory Authority website shows that savings for some types of motor vehicles under the 2017 CTP scheme exceed $60. The State Insurance Regulatory Authority website states that the average saving under this scheme has been $129. Service NSW advised that these advertisements were designed for regional markets and that it used different advertisements for metropolitan areas which contained different amounts of savings.

Some advertisements in phase 2 stated, ‘My Toll Relief save up to $700’. The Service NSW website states that drivers can obtain free vehicle registration if they have spent $1,352 or more in tolls in the previous financial year. The cost of registration for some vehicles exceeds $700. This means the savings detailed in the advertisement were lower than what some customers could actually save.

NSW Rural Fire Service conducted the 'How FireProof Is Your Plan?' (Fireproof) campaign. The Fireproof campaign is a three-year campaign which ran in 2018–19 (year one), 2019–20 (year two) and is planned for 2020–21 (year three). This audit examined year two of the campaign (2019–20).

The Fireproof campaign is a public safety campaign encouraging people to plan and prepare for bush fires across the summer period. The campaign aims to improve the quality of bush fire planning and preparation in the community and decrease the impact of fires on the community when they occur.

Campaign materials we reviewed did not breach section 6 of the Act

The audit team reviewed campaign materials developed as part of the paid advertising campaign for example radio advertisements, television commercials and digital displays. The audit team did not review the use of social media outside paid social media content as section four of the Act defines government advertising as the dissemination of information which is funded by or on behalf of a government agency. Examples of campaign materials are shown in Appendix two.

Section 6 of the Act prohibits political advertising as part of a government advertising campaign. A government advertising campaign must not:

• be designed to influence (directly or indirectly) support for a political party
• contain the name, voice or image of a minister, a member of parliament or a candidate nominated for election to parliament
• contain the name, logo, slogan or any other reference to a political party.

The audit found no breaches of section 6 of the Act in the campaign material we reviewed. 

NSW Rural Fire Service did not set targets for the second year of the campaign

The second year of the Fireproof campaign (2019–20) had the same objectives as the first year of the campaign (2018–19), however no specific targets were set for the second year. The advertising submission for the first year of the campaign (2018–19) details the targets for each objective as an increase of ten per cent against the baseline data to be achieved by March 2021, at the end of the three-year campaign.

The second year of the Fireproof campaign (2019–20) was one of the first campaigns approved under the new budget and peer review processes introduced by the Department of Customer Service in 2019–20. The new process for peer review introduced a new template for campaign submissions. The former template for campaign submissions contained more prompts for agencies to ensure the submission contained sufficient detail of campaign objectives, baseline measures, targets, dates for measurement and detail on how they would measure objectives. Despite this, the peer review process should have identified that NSW Rural Fire Service did not set targets for the second year of the campaign.

The 2016 Guidelines for Implementing NSW Government Evaluation Framework for Advertising and Communications requires campaign objectives to be SMART (specific, measurable, achievable, realistic and timed). NSW Rural Fire Service did not meet this requirement for year two of the Fireproof campaign.

Post-campaign evaluations showed increases against four out of five objectives, however there were no specific targets

NSW Rural Fire Service set three campaign objectives at the time it submitted the second year of the campaign (2019–20) to the Department of Customer Service for peer review. However, the post-campaign effectiveness report submitted to the Department of Customer Service measured campaign effectiveness against five campaign objectives. The objectives in the post-campaign effectiveness report were the same objectives set for the first year of the campaign, which is appropriate as this was a repeat campaign.

NSW Rural Fire Service achieved increases against four of their five objectives. However, as noted above there were no specific targets (such as percentage increases) against which performance of the 2019–20 campaign could be measured. Despite this, at the end of the second year, the Fireproof campaign had already achieved some of the targets that NSW Rural Fire Service had set for the end of the third year of the campaign. The post-campaign research showed that both audience recall and exposure to the campaign increased significantly from the prior year. The campaign objectives and results are shown in Exhibit 8.

For those people who already have a bush fire plan, the campaign aimed to increase the number of those plans which have included two or more elements from the Guide to Making a Bush Fire Survival Plan. Elements from the Guide to Making A Bush Fire Survival Plan include actions such as deciding what to take with you if you leave, ensuring you have the right equipment for defending your home and allocating responsibilities to members of a household. The post-campaign evaluation showed that the campaign did not achieve an increase against this objective for people who planned to stay and defend their property rather than leave.

Exhibit 8: Campaign objectives and results

Campaign objectives	Does the post-campaign evaluation show increases against the objective?
1. Continue to increase the number of people that have discussed and/or written a plan with regards to what they will do in the event of a fire.
2. Of those who indicate they have a plan, increase the number of people who have included two or more elements from the Guide to Making a Bush Fire Survival Plan:
for those who plan to leave
for those who plan to stay and defend.
3. Increase the frequency in completing preparation activities around a person’s property.
4. Increase the number of people who correctly assess it is their responsibility to complete preparation activities and enact their plan without direct intervention from emergency services.
5. Visits to MyFirePlan website.

Key

Yes

No

NSW Rural Fire Service achieved cost efficiencies by reusing creative content developed in the first year of the campaign

Total creative and production costs incurred in year one of the campaign were $1.08 million. Rather than commissioning new creative materials, NSW Rural Fire Service re-used the same creative content in year two of the campaign. NSW Rural Fire Service incurred $100,000 in creative and production costs in year two of the campaign and achieved cost-efficiencies by reusing the same creative developed in the prior year.

NSW Rural Fire Service allowed sufficient time for cost-efficient media placement and received free media placements

The Department of Customer Service advises agencies to work with media contacts to book media in advance to ensure a cost-efficient placement. Prior to 2019–20, the Department of Premier and Cabinet provided suggested timeframes for agencies to book media as part of the peer review process. For example, it advised agencies to book television six to 12 weeks in advance and book radio advertising two to eight weeks in advance. NSW Rural Fire Service allowed sufficient time for a cost-efficient media placement.

NSW Rural Fire Service received $4 million of free advertising time and space donated by media companies due to the extent and impact of the 2019–20 fire season.

The cost benefit analysis (CBA) did not assess the costs and benefits of alternatives to advertising

Under the Government Advertising Act 2011, agencies are required to prepare a CBA when the cost of the campaign is likely to exceed $1 million. As part of the CBA, the Government Advertising Guidelines require agencies to consider options other than advertising to achieve the desired objective including a comparison of costs and benefits.

The CBA for the Fireproof campaign (year two) notes that the proposed campaign is one component of a broader community engagement strategy which has been developed over time and is based on research and evaluation. The CBA considers two options to achieve the objectives of the campaign. The first option is community engagement activities without an advertising campaign and the second option is community engagement activities alongside an advertising campaign. The CBA does not identify and assess the costs and benefits of both of the options in order to assess the most cost-efficient option.

This is a repeat finding from two previous government advertising audits. The report ‘Government Advertising: 2015–16 and 2016–17’ found that both agencies subject to the audit did not meet the requirements in the guidelines to consider alternatives to advertising. The report made a recommendation to the Department of Premier and Cabinet to work with Treasury to ensure the requirements of the guidelines are fully reflected in the 'Cost-Benefit Analysis Framework for Government Advertising and Information Campaigns'. The report ‘Government advertising 2017–18’ found that one agency subject to the audit did not identify to what extent the benefits could be achieved without advertising, nor did it consider alternatives to advertising which could achieve the same impact as the advertising campaign.

Appendix one – Responses from agencies

Appendix two – About the campaigns

Appendix three – About the audit

Appendix four – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #342 - released 19 November 2020

The Auditor-General for New South Wales, Margaret Crawford, released a report today examining whether the Department of Communities and Justice had effective governance and partnership arrangements in place to deliver ‘Their Futures Matter’.

Their Futures Matter was intended to place vulnerable children and families at the heart of services, and direct investment to where funding and programs deliver the greatest social and economic benefits. It was a four-year whole-of-government reform in response to the 2015 Tune Review of out-of-home care.

The Auditor-General found that while important foundations were put in place, and new programs trialled, the key objective to establish an evidence-based whole-of-government early intervention approach for vulnerable children and families in NSW was not achieved.

Governance and cross-agency partnership arrangements to deliver Their Futures Matter were found to be ineffective. 'Their Futures Matter lacked mechanisms to secure cross portfolio buy‑in and did not have authority to drive reprioritisation of government investment', the Auditor-General said.

At the reform’s close, the majority of around $380 million in investment funding remains tied to existing agency programs, with limited evidence of their comparative effectiveness or alignment with Their Futures Matter policy objectives. The reform concluded on 30 June 2020 without a strategy or plan in place to achieve its intent.

The Auditor-General made four recommendations to the Department of Communities and Justice, aimed at improving implementation of outstanding objectives, revising governance arrangements, and utilising the new human services data set to address the intent of the reform. However, these recommendations respond only in part to the findings of the audit.

According to the Auditor-General, ‘Cross-portfolio leadership and action is required to ensure a whole-of-government response to delivering the objectives of Their Futures Matter to improve outcomes for vulnerable children, young people and their families in New South Wales.’

Read full report (PDF)

In 2016, the NSW Government launched 'Their Futures Matter' (TFM) - a whole-of-government reform aimed at delivering improved outcomes for vulnerable children, young people and their families. TFM was the government's key response to the 2015 Independent Review of Out of Home Care in New South Wales (known as 'the Tune Review').

The Tune Review found that, despite previous child protection reforms, the out of home care system was ineffective and unsustainable. It highlighted that the system was not client-centred and was failing to improve the long-term outcomes for vulnerable children and families. The review found that the greatest proportion of relevant expenditure was made in out of home care service delivery rather than in evidence-based early intervention strategies to support children and families when vulnerabilities first become evident to government services (such as missed school days or presentations to health services).

The then Department of Family and Community Services (FACS) designed the TFM reform initiatives, in consultation with central and human services agencies. A cross-agency board, senior officers group, and a new unit in the FACS cluster were established to drive the implementation of TFM. In the 2016–17 Budget, the government allocated $190 million over four years (2016–17 to 2019–20) to the reform. This resourced the design and commissioning of evidence-based pilots, data analytics work, staffing for the implementation unit and secretariat support for the board and cross-agency collaboration.

As part of the TFM reform, the Department of Premier and Cabinet, NSW Treasury and partnering agencies (NSW Health, Department of Education and Department of Justice) identified various existing programs that targeted vulnerable children and families (such as the preceding whole-of-government ‘Keep Them Safe’ reform coming to an end in June 2020). Funding for these programs, totalling $381 million in 2019–20, was combined to form a nominal ‘investment pool’. The government intended that the TFM Implementation Board would use this pool to direct and prioritise resource allocation to evidence-based interventions for vulnerable children and families in NSW.

This audit assessed whether TFM had effective governance and partnership arrangements in place to enable an evidence-based early intervention investment approach for vulnerable children and families in NSW. We addressed the audit objective with the following audit questions:

• Was the TFM reform driven by effective governance arrangements?
• Was the TFM reform supported by effective cross-agency collaboration?
• Has the TFM reform generated an evidence base to inform a cross-agency investment approach in the future?

The audit did not seek to assess the outcomes for children, young people and families achieved by TFM programs and projects.

Their Futures Matter (TFM) is a whole-of-government reform to deliver improved outcomes for vulnerable children, young people and their families.

Supported by a cross-agency TFM Board, and the TFM Unit in the then Department of Family and Community Services (FACS), the reform aimed to develop whole-of-government evidence-based early intervention investment approaches for vulnerable children and families in NSW.

Governance refers to the structures, systems and practices that an organisation has in place to:

• assign decision-making authorities and establish the organisation's strategic direction
• oversee the delivery of its services, the implementation of its policies, and the monitoring and mitigation of its key risks
• report on its performance in achieving intended results, and drive ongoing improvements.

We examined whether the TFM reform was driven by effective governance arrangements and cross-agency collaboration.

The reform agenda and timeframe set down for Their Futures Matter (TFM) were ambitious. This chapter assesses whether the TFM Board and TFM Unit had the capability, capacity and clout within government to deliver the reform agenda.

Creating a robust evidence base was important for Their Futures Matter, in order to:

• identify effective intervention strategies to improve supports and outcomes for vulnerable children and families
• make efficient use of taxpayer money to assist the maximum number of vulnerable children and families
• inform the investment-based approach for future funding allocation.

This chapter assesses whether the TFM reform has developed an evidence base to inform cross-agency investment decisions.

Appendix one – Response from agency

Appendix two – TFM governance entities

Appendix three – TFM Human Services Data Set

Appendix four – TFM pilot programs

Appendix five – About the audit

Appendix six – Performance auditing

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.

Parliamentary reference - Report number #337 - released 24 July 2020

This report contains findings on the results of financial audits of NSW universities for the year ended 31 December 2019.

All ten NSW universities received unqualified audit opinions. The 2019 financial results for universities are reported as at 31 December and reflect results from operations before the impact of the COVID‑19 pandemic.

The combined revenues for all NSW universities increased by $381 million to $11.4 billion in 2019, driven by increases in student revenues. Revenue from overseas students continued to grow faster than that from domestic students and contributed $3.6 billion in course fees to NSW universities in 2019.

Overseas students from the top three countries of origin, being China, India and Nepal, represented 72.4 per cent of all enrolments of overseas students and 65.4 per cent of all overseas student revenues for 2019. Revenue from students from these three countries comprised 40.9 per cent of total student revenues for all NSW universities, creating a considerable concentration risk for NSW universities.

The COVID‑19 pandemic may significantly impact the financial results of NSW universities in 2020. NSW universities provided data on COVID‑19 impacted student enrolments for semester one 2020. Overall numbers of student enrolments in semester one 2020 were 5.8 per cent beneath projections. Overseas student enrolments were 13.8 per cent beneath expectations and domestic student enrolments were 2.4 per cent below expectations.

The report makes recommendations to the NSW universities, aimed at strengthening controls over information technology, cyber security, validating published performance information, procurement practices and the oversight of their overseas controlled entities' legal and policy compliance functions.

Read full report (PDF)

This report analyses the results of our audits of the financial statements of the ten NSW universities for the year ended 31 December 2019. The table below summarises our key observations.

1. Financial reporting

2. Internal controls and governance

3. Teaching and research

This report provides Parliament with the results of our financial audits of New South Wales universities and their controlled entities in 2019, including our analysis, observations and recommendations in the following areas:

• financial reporting
• internal controls and governance
• teaching and research.

Financial reporting is an important element of governance. Confidence and transparency in university sector decision making are enhanced when financial reporting is accurate and timely.

This chapter outlines our audit observations on the financial reporting of NSW universities for 2019.

Appropriate and robust internal controls help reduce risks associated with managing finances, compliance and administration of NSW universities.

This chapter outlines the internal controls related observations and insights across NSW universities for 2019, including overall trends in findings, level of risk and implications.

Our audits do not review all aspects of internal controls and governance every year. The more significant issues and risks are included in this chapter. These along with the less significant ones are reported to universities for them to address.

Universities' primary objectives are teaching and research. They invest most of their resources to achieve quality outcomes in academia and student experience. Universities have committed to achieving certain government targets and compete to advance their reputation and international and Australian rankings.

This chapter outlines teaching and research outcomes for NSW universities for 2019.

Appendix one – List of 2019 recommendations

Appendix two – Status of 2018 recommendations

Appendix three – NSW universities’ controlled entities and associated entities

Copyright notice

© Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.