No agency adopted a comprehensive approach to addressing public inquiry recommendations

Agencies addressed recommendations from public inquiries with varying levels of formality. Some agencies maintained registers to track and monitor implementation progress. These were mostly recent developments over the last three or four years. Other agencies incorporated recommendation implementation into business as usual practices.

No agency maintained a comprehensive approach for all public inquiries. This increases the risk that recommendations from public inquiries are overlooked or not addressed with priority.

Agencies nominated milestone dates, risk and priority for only a third of recommendations

Only a third of recommendations we reviewed had target completion or milestone dates attached. This was mostly for audit and coronial inquiry recommendations. Similarly, agencies did not routinely nominate priority and risk for recommendations.

This makes it difficult for agencies to effectively triage recommendations for action, to track progress in implementing them or identify where escalation is required. The agencies advised us that they had addressed most of the recommendations within the scope of this audit.

For two-thirds of the reviewed recommendations, agencies did not test and verify that recommendations have been implemented as intended

Agencies also did not always take steps to verify or challenge internal advice that recommendations were progressing or implemented as intended. This means that agencies cannot readily identify whether actions taken to give effect to recommendations are in line with the intent and real-world outcomes sought.

Only around a third of recommendations we reviewed were tested in this way, and mainly for audit recommendations. This introduces risks that disaster response gaps identified in inquiries are not fully or appropriately addressed and recur in future disasters.

Agencies do not publish a summary view of public inquiry recommendation implementation

In our 2015 'Implementing performance audit recommendations report' we highlighted the annual report as one possible avenue for agencies to share information about the status of performance audit recommendations. No agency did this for the three performance audits we reviewed for this audit. Nor did any agency take this approach for recommendations from the other types of public inquiries within the audit scope. Doing so would improve transparency and accountability.

One factor preventing agencies publishing this information is that it has not been consolidated within agencies and maintained adequately to make the publication of summary information straightforward.

Without published information, it is difficult for the public to determine what actions have been taken to give effect to public inquiry recommendations, and whether these have led to improvements in agency capacity to prevent, prepare for and respond to natural disasters.

By September 2021, the Department of Planning, Industry and Environment (NSW National Parks and Wildlife Services), Fire and Rescue NSW, Resilience NSW, NSW Rural Fire Service and the NSW State Emergency Service should:

1. establish an approach to tracking the implementation of accepted recommendations from public inquiries, with the following requirements at a minimum:

a) a Senior Executive is accountable for the approach

b) a Senior Executive action officer is accountable for implementation of accepted recommendations

c) early assessment is undertaken to determine the appropriate processes and structures that should apply to addressing each accepted recommendation

d) priority, risk and completion dates are nominated for each action to address public inquiry accepted recommendations

e) changes to priority, risk and completion dates are counter-signed by the Senior Executive responsible for the approach

f) the Senior Executive responsible for the approach collects updates from action officers quarterly

g) there is regular reporting to an appropriate Senior Executive management group that highlights at-risk items

2. formally acquit accepted recommendations from public inquiries based on evidence, including an assessment of whether the action taken has met the intent of the commitment to the accepted recommendation, and whether ongoing monitoring is required to embed changes made

3. publish consolidated, summary information at least every 12 months on progress made to implement accepted recommendations from public inquiries including:

a) action nominated to address accepted public inquiry recommendations

b) expected completion date for action to address accepted public inquiry recommendations

c) status of action to address accepted public inquiry recommendations (e.g. on track, delayed, implemented or not implemented)

d) any changes made to actions and/or expected completion dates, and reasons for these.

By December 2021, Resilience NSW should:

4. establish and commence operating the central accountability mechanism recommended by the NSW Bushfire Inquiry

5. coordinate with all agencies affected by public inquiries focused on disaster response and recovery to participate in the central accountability mechanism recommended by the NSW Bushfire Inquiry.

On an ongoing basis, the Department of Planning, Industry and Environment (NSW National Parks and Wildlife Services), Fire and Rescue NSW, Resilience NSW, NSW Rural Fire Service and the NSW State Emergency Service should:

6. brief their Audit and Risk Committee on the risk implications of public inquiry findings, accepted recommendations and proposed actions to address recommendations for the agency.

Major natural disasters are usually followed by inquiries to determine the cause; the effectiveness of the preparedness, response and recovery efforts; and to identify areas of improvement. We reviewed the range of public inquiries in NSW over the last ten years that affected the audited agencies' operational response to bushfires, floods and storms.

This included 17 public inquiries that made recommendations that were accepted, partially accepted or accepted in principle by the NSW Government or the agencies under review.

The 17 public inquiries made 305 recommendations to NSW Government agencies. Of these, 224 recommendations were made to agencies audited in this report, and 191 of those recommendations were supported, supported in principle or partially supported. The 191 recommendations covered a wide range of themes.

Exhibit 1 shows characteristics of the 17 in scope public inquiries and recommendations, including agency reported implementation status as at March 2021. Bushfire was a focus of seven of the inquiries, storm and flood a focus of five and workforce issues were the focus of the remaining five public inquiries.

Agencies reported that they had implemented most of the recommendations from the public inquiries in scope for this audit. See Appendix five for more details on recommendation implementation status.

Following the 2019–20 bushfire season, two major inquiries explored aspects of the NSW emergency response:

• NSW Bushfire Inquiry
• Commonwealth Royal Commission into National Natural Disaster Arrangements.

Agency responses to these two inquiries are out of scope for this audit, as not enough time has elapsed to evaluate those responses. However, both of these inquiries observed the significant volume of previous reviews of bushfires and other disasters. Both inquiries also noted the difficulty in establishing whether and how recommendations from these previous inquiries had been implemented, from publicly available information.

The NSW Bushfire Inquiry reported in August 2020 and made a recommendation to NSW Government:

That, in order to ensure recommendations accepted by the government are implemented in a timely and transparent manner government establish a central accountability mechanism to track implementation of recommendations from bushfire related reviews and inquiries and consider expanding this to other areas.

The NSW Government accepted in principle all recommendations from the NSW Bushfire Inquiry. Subsequently, NSW Parliament legislated for the Emergency Services Minister to provide updates on all recommendations from this inquiry, every three months.

The Royal Commission into National Natural Disaster Arrangements reported in October 2020. In conduct, the Royal Commission asked all states and territories to submit information to it on how the findings and recommendations of previous inquiries had been addressed, but found it difficult to assess the implementation status of some recommendations.

The Royal Commission was supportive of the external review and assurance agencies that have been established in Victoria and Queensland, Inspectors-General of Emergency Management. These are independent statutory roles that undertake objective reviews, evaluations and assessments of emergency management arrangements.

The Victorian and Queensland Inspectors-General also examine and provide assurance on the sector's performance, capacity and capability to plan for, respond to and recover from emergencies. This includes monitoring the implementation of recommendations and actions identified through reviews to ensure they are effective and sustainable in the long-term. The Royal Commission recommended that:

Each state and territory government should establish an independent accountability and assurance mechanism to promote continuous improvement and best practice in natural disaster arrangements.

The NSW Government has committed to working with the Australian Government to respond to the Royal Commission's recommendations, complementing its response to the recommendations from the NSW Bushfire Inquiry.

Audited agencies

For this audit we reviewed five agencies:

• Fire and Rescue NSW
• NSW National Parks and Wildlife Services (now a division of the Department of Planning, Industry and Environment)
• NSW Rural Fire Service
• NSW State Emergency Service
• Resilience NSW (formerly the Ministry for Police and Emergency Services; and the Office of Emergency Management).

These agencies were selected as they are or were:

• either a combat agency charged with responding to floods, storms and bushfires; or they play a key coordinating role in recovery responses to these disaster events
• responsible for addressing related public inquiry recommendations.

The following governance entities are relevant to how the audited agencies respond to public inquiry recommendations, but were not the subject of our audit.

Audit and Risk Committees

NSW Treasury Policy requires agencies to establish an Audit and Risk Committee. The guidance that applied during the audit period was Treasury Policy TPP15-03, but this has recently been replaced by TPP20-08 with the same requirement.

Audit and Risk Committees are advisory committees with independent members, that provide assistance to the head of an agency on the agency's governance processes, risk management and control frameworks as well as external accountability obligations.

Emergency Services Board of Commissioners

The Emergency Services Board of Commissioners is an advisory body to the Emergency Services Minister. Members include the Commissioners of Fire and Rescue NSW, Resilience NSW, the NSW Rural Fire Service and the NSW State Emergency Service. Resilience NSW (and its predecessor organisation, the Office of Emergency Management) provides secretariat support for the Board of Commissioners and tracks recommendations for which the Board is responsible.

State Emergency Management Committee

The State Emergency Management Committee is a statutory advisory body to the Emergency Services Minister. Its members include heads of all emergency services agencies. The State Emergency Management Committee plays a lead role in coordinating efforts to prevent, prepare, respond to and recover from emergencies and disasters of all kinds and across all agencies. In November 2020, and in response to the 2019–20 bushfires, Parliament passed legislation giving the State Emergency Management Committee an explicit function to promote continuous improvement and conduct reviews as required.

See Appendix three for a list of inquiries and responsible agencies.

The audit objective was to assess how effective NSW emergency response agencies are in addressing accepted recommendations from public inquiries. We addressed the audit objective by answering two questions:

• Do agencies have effective governance arrangements in place to respond to, monitor and implement accepted recommendations from public reviews and inquiries?
• Do agencies provide timely and accurate information on the implementation of accepted inquiry recommendations to senior decision makers and the public?

This audit did not assess the initial advice to government or agency executives on whether to accept public inquiry recommendations, nor the individual actions taken to implement specific recommendations.

More information on the audit approach is in Appendix six.

For two-thirds of the reviewed recommendations, agencies did not test and verify that recommendations have been implemented as intended

Acquitting recommendations involves checking evidence that the recommendations have been done and that action aligns with the intent of the recommendation. This provides agencies with assurance and is important to inform stakeholders that efforts taken will address the gaps identified by inquiries and improve disaster preparedness and response.

Formal acquittal of recommendations provides an opportunity to:

• verify whether action has been taken as planned, by examining relevant evidence
• determine whether the action taken meets the intent of the recommendation and the real-world outcome sought
• assess whether the action taken addresses similar or repeat recommendations
• determine whether broader lessons may be drawn for continuous improvement
• determine whether any ongoing reporting or action is needed to embed the changes desired.

Queensland and Victoria have established independent assurance functions that monitor the status of emergency management recommendations. In the absence of a similar independent assurance function in NSW, we expected that agencies would have processes in place to verify and test the implementation of accepted recommendations.

However, none of the audited agencies had a consistently applied approach to reviewing, closing off or acquitting recommendations from public inquiries based on evidence. Only 34 per cent of recommendations we reviewed were formally challenged as part of an acquittal process. These recommendations were mostly from audit and independent inquiries.

Where this was done, this involved a Senior Executive examining evidence of action taken and considering whether this was in line with the recommendation's intent before signing it off. Other actions that did not require changes to policies and procedures with delegated approvals were not always formally acquitted.

For instance, several recommendations from the Wambelong and Warrumbungle coronial inquiries required NSW National Parks and Wildlife Service to consult with stakeholders on certain matters. NSW National Parks and Wildlife Service report that this occurred, but were not required to acquit this outcome at the time. However, in early 2018 NSW National Parks and Wildlife Services did conduct a subsequent appraisal of recommendation implementation on the five-year anniversary of the fires which found that most of the recommendations were implemented.

The NSW State Emergency Service has set business rules around all audit reports, including performance audits. Over the period of review, the NSW State Emergency Service also established an Operational Improvement Database that tracks after-action reviews, coronial inquiries and other selected reviews. Nevertheless, a recent internal audit report identified inconsistencies in how recommendations were closed off or acquitted. In response, the NSW State Emergency Service has developed and commenced a new framework to support improved monitoring and acquittal of recommendations including a requirement for evidence besides the advice of relevant officers.

Agencies identified that six of the 191 accepted recommendations of the inquiries within the audit scope were implemented but require ongoing action or monitoring. The meaning or significance of ongoing action or monitoring was not defined by any business rule and exposes agencies to the risk that recommendations are not fully embedded in an organisation.

The lack of acquittal for many recommendations means that in many instances agencies and key stakeholders cannot know for certain that recommendations have been implemented in accordance with the original commitment. This creates a risk that the public safety gaps identified in inquiries and which recommendations are drafted to address are not fully addressed.

All agencies assigned a staff member or unit to each accepted recommendation

All agencies assigned responsibility for implementing recommendations to a relevant staff member or branch with an appropriate level of responsibility to carry out the proposed action.

No agency used a central governance unit or process to coordinate responsibilities for addressing recommendations. This reduces the visibility across the organisation of who is responsible, and increases the risk that different public inquiry recommendations will be treated and managed in silos.

Agencies did not develop detailed plans to implement most accepted recommendations

Public inquiry recommendations vary in scale and scope. Some recommendations require significant changes, such as to legislation, whereas other recommendations require comparatively small changes, such as to internal processes. Accordingly, it is understandable that agencies would develop plans of varying detail in response.

Implementation and action plans varied by type of inquiry and the substance of actions taken for giving effect to recommendations. For example, Fire and Rescue NSW established a formal project to address recommendations from the ‘Fitness of firefighters’ performance audit report. Similarly, the Emergency Services Board of Commissioners established a series of projects to respond to recommendations from the Bega Valley fire independent inquiry and Emergency services parliamentary inquiry.

Most other recommendations for audited agencies were assigned to staff or a business area as part of day-to-day operations. The decisions about whether to manage the implementation of recommendations as a project were appropriate.

Managing actions as a project significantly increases the reporting and governance burden and may not be needed in every instance.

However, we found that not managing recommendation implementation as a project typically meant that nominating key milestone dates, priority, risk, regular reporting and oversight for key actions was not as common. Overall, agencies only nominated milestones dates for 34 per cent of the recommendations we reviewed. Recommendations from audit and coronial inquiries made up the bulk of these instances. The absence of milestones reduces accountability and exposes agencies to risks that implementation issues are not escalated appropriately in responding to recommendations from public inquiries.

Of the 22 recommendations where agencies reported that implementation is still in progress, agencies had nominated milestone dates for just six recommendations; and four recommendations remain outstanding after five years. Exhibit 2 below shows the number of recommendations reported by agencies as still being addressed, for which agencies have or have not nominated milestones. This shows that 16 of the 22 recommendations still being implemented did not have milestone dates nominated.

Appendix four provides more detail for these recommendations and responsible agencies.

Resilience NSW reports that there are three recommendations from recovery reports issued five years ago that remain outstanding. All relate to multi-agency projects. Multi-agency projects, especially inter-jurisdictional projects, are more complicated to implement. However, this also highlights that more detailed monitoring of recommendations could be valuable to keep these actions on track.

Several more recommendations for which Resilience NSW were responsible were only recently closed out following the 2019–20 bushfire season, with that significant event providing the apparent impetus for doing so. There are a further two recommendations that agencies report they have not yet commenced – recommendations two and three of the Bega Valley fires independent review. The status of these recommendations is reported to the Emergency Services Board of Commissioners.

Once set, actions to address accepted recommendations from public inquiries were rarely altered

We identified only two instances where agencies adjusted their approaches to implementing recommendations after commencing implementation. These two were in response to a subsequent event, and after further consideration of an accepted recommendation.

Few agencies had a formal process to review recommendation implementation actions at key stages, to consider their continued relevance. However, staff we spoke to across the agencies commented that such a review point would be a useful feature to ensure that actions, and their risk and priority ratings, remain appropriate. Such a review might also reveal whether more resources are required to address a recommendation than initially planned.

Agencies look for practice improvements from other jurisdictions

Agencies reported participation in a variety of cross-agency forums, such as through the Australasian Fire and Emergency Service Authorities Council (AFAC), to identify good practice and developments from other jurisdictions and settings that may be relevant. Similarly, agencies reported closely monitoring public inquiries from other jurisdictions and disaster types for insights that could be relevant to their own practices.

Resilience NSW is responsible for administering the NSW Lessons Management Framework for the NSW Emergency Management Sector. Published in 2019, the 2019–20 bushfires was the first disaster for which this new framework was in place. The NSW Lessons Management Framework complements existing lessons management processes in place at individual agencies. These provide further opportunities to reflect on recommendations and determine whether adjustment is required to the action initially planned. 

No agency had a comprehensive approach to addressing public inquiry recommendations

None of the audited agencies maintained a comprehensive approach that drove or tracked the implementation of recommendations from all types of inquiries.

Treasury Policy requires agencies to monitor the implementation of performance audit recommendations but not other types of inquiries, and it does not require agencies to use a formal register to track performance audits. A register is not essential to responding to, monitoring and implementing recommendations from public inquiries. However, a well-maintained register with clearly articulated business rules has the following benefits. It:

• allows for alignment, overlap or conflict between recommendations or agency actions to be identified
• encourages early assessment of the relative priority, risk rating, governance arrangements, and monitoring requirements that should be used for addressing different types of inquiries and recommendations
• specifies expected milestones and timeframes, and highlights delays for escalation
• provides visibility for more consistent and routine reporting on public commitments.

The NSW State Emergency Service maintained two registers: one for audit, including performance audits, and another for operational-focused inquiries and lessons management. However the rules for the latter were not articulated as business rules and procedures for entering and updating information varied.

Fire and Rescue NSW and the NSW Rural Fire Service also tracked and monitored the implementation of performance audit recommendations. However, this was not done through the audit register. Fire and Rescue NSW has tracked coronial inquiries through a dedicated register since 2018. The NSW Rural Fire Service maintains a Master Audit and Review register to track operational audits and inquiries, but this does not track all internal or performance audits.

As noted in the introduction, the Emergency Services Board of Commissioners is an advisory body to the Emergency Services Minister and Resilience NSW provides the secretariat to the Board. The Board of Commissioners was assigned responsibility for cross-agency recommendations from two of the inquiries in scope for this audit. However, neither Resilience NSW nor its predecessor agencies maintain a register for operational or non-operational inquiries outside of those that are the responsibility of the Board of Commissioners.

The NSW National Parks and Wildlife Service also did not maintain a register or similar for tracking recommendations from the two inquiries affecting them within the audit scope. However, since 2017 the NSW National Parks and Wildlife Service has established a database of coronial and parliamentary inquiries. The NSW National Parks and Wildlife Service plans to use this database to track and monitor future public coronial and parliamentary inquiries and to expand this database across the Department of Planning, Industry and Environment.

Agencies included some proposed actions to address relevant recommendations in planning documents

Corporate planning identifies agencies' key priorities and enhances accountability over actions taken. Some, but not all, actions to address recommendations from public inquiries were incorporated by agencies into regular business planning and other strategic documents where they were relevant.

The NSW Rural Fire Service, for instance, maintained a detailed system of business planning and related reporting over the period of review. Several actions to address public inquiry recommendations were included in this system.

Similarly, the NSW National Parks and Wildlife Service maintained detailed branch plans incorporating many of the actions to address recommendations from its two inquiries in scope for this audit. However, this system was primarily used to plan resource allocations, rather than as a system of accountability for tracking recommendation implementation.

Incorporating relevant actions to address recommendations into corporate planning can increase the visibility of recommendations and the likelihood they will be actioned. It should be noted that this will not always be appropriate as corporate plans need to focus on priorities rather than all, or a large sub-set of, the activities of an agency.

Reporting to senior decision makers could be enhanced

All agencies kept senior management informed, from time to time, of the progress in implementing accepted recommendations from public inquiries. Agencies that maintained registers for certain types of public inquiries had more routine reporting to senior managers on recommendation implementation. However, as noted in the previous chapter, consistent weaknesses in how agencies verified that recommendations were addressed, and failure to nominate recommendation milestone dates, priorities and risks, reduced the potential for reporting to maintain senior management focus on implementation.

Regular reporting took the form of dashboard-style reporting highlighting overdue items. This style of reporting allows senior management to see at a glance how agencies are performing in implementing public commitments, and highlights where intervention may be necessary. Reports like this can be produced quickly for consideration and – with sufficient attention to business rules for maintaining the register – accurately. Resilience NSW, for instance, provides the Emergency Services Board of Commissioners with a regular item noting the summary status of various projects, including responses to public inquiry recommendations, that allows the Board to quickly identify at risk projects.

However, this kind of reporting can only be effective if agencies nominate key milestones, priority and risk ratings for recommendation implementation and this usually does not occur.

Other reporting to senior management was ad hoc, event centred and often coincided with delegated approvals required to implement a certain action. For example, the NSW National Parks and Wildlife Service addressed several recommendations from the Warrumbungle coronial inquiry through updates to its 2016–17 Fire Management Manual. As these Manual updates were finalised, senior management were informed through the approvals sought for the change.

Agencies are appropriately briefing their Audit and Risk Committees on performance audits

The role of the Audit and Risk Committee (ARC) with respect to performance audits is clear. Treasury Policy requires agencies to track and monitor the implementation of performance audit recommendations. The policy requires that agencies maintain a register for internal audits but does not require that same approach for performance and other external audits. The ARC reviews the agency's management plans and monitors the implementation of performance audit recommendations. This role of the ARC was also reviewed in our 2015 performance audit on ‘Implementing performance audit recommendations’.

Three agencies and four performance audits were in scope for this report:

• Fire and Rescue NSW responding to the Fitness of firefighters and Preventing and managing worker injuries performance audits
• NSW Rural Fire Service responding to the Fitness of firefighters performance audit
• NSW State Emergency Service responding to the SES management of volunteers performance audit.

These agencies tracked and monitored recommendations from these inquiries consistent with Treasury policy requirements, and provided information to their ARCs to allow for oversight.

Agencies do not consistently present information on relevant risks from other public inquiries to their Audit and Risk Committees

Treasury policy requires agencies to maintain a risk management framework. A risk management framework includes processes to identify risks to the agency. Coronial, parliamentary and other inquiry types all identify potential strategic, operational, financial and reputational risks for the organisation. While the head of the agency is ultimately responsible and accountable for risk management, the ARC has an important advisory role. However, agencies do not consistently present information on the agency risks highlighted by public inquiries to the ARCs, apart from performance audits.

The NSW National Parks and Wildlife Service and the NSW Rural Fire Service are involved with a high number of coronial proceedings. Accordingly both agencies presented regular standing items to their ARCs on current and upcoming coronial proceedings, but not on other types of public inquiries within the audit scope.

The other agencies presented the findings and recommendations of public inquiries other than performance audits to their ARCs as part of periodic strategic overviews of the agencies' operations. A more consistent assessment of the risk implications of public inquiries, and briefing of the ARCs on these, could enhance agencies' risk management and provide better information to the ARC for its advisory role.

Agencies do not regularly report on public inquiry recommendation implementation in their annual reports

Public reporting on the implementation of public inquiry recommendations is an important mechanism for transparency and accountability. Some actions to address public inquiry recommendations are, by their nature, public - such as changes to legislation, policy administration or building codes. Other actions are less obvious. Public reporting brings this information together for consideration and appraisal, and enhances accountability for what are public commitments to action.

The Annual Reports (Departments) Regulation 2015 was in place for the period of review for this audit. It requires agencies to report on:

The nature and extent of performance review practices and of improvements in organisational achievements as assessed by both internal and external performance reviews.

Benefits achieved as a result of management and strategy reviews.

A description of management improvement plans adopted by the Department and achievements in reaching previous targets.

In our 2015 report on ‘Implementing performance audit recommendations’, to support this guidance, we recommended that agencies publish progress against performance audit recommendations in the agencies' annual reports.

We recommended that this take the form of a table showing the number of recommendations on track, closed or delayed and the proposed implementation dates. None of the agencies have done so for the performance audits included in this audit.

Of the reviewed agencies, only the NSW Rural Fire Service and the NSW State Emergency Service provided limited discussion of the other types of inquiries in scope for this report, including recommendations from our performance audits.

More detailed reporting on the status and outcomes of planned actions to address accepted recommendations provides opportunities for agencies to highlight improvements in practice. It also enables agencies to show the public how they are taking on lessons from inquiries, and strengthening their capacity to prevent, prepare for, respond to and recover from natural disasters.

No agency published summary status updates of public inquiry recommendation implementation

The Government Information (Public Access) Act 2009 (GIPA Act) permits and encourages NSW Government agencies to proactively identify information for public release. Consistent with this, all of the emergency response agencies for this audit have an annual cycle in place to identify information that might be made available. Under open government policies, agencies are encouraged to first consider commercially useful and other high-value datasets for public release.

None of the agencies reviewed had considered making public summary information about how they were implementing accepted recommendations from public inquiries. In part this is a consequence of the fragmented way in which public inquiry recommendations are addressed, which hinders the publication of this data.