3. Planning and procurement
Chapter 1. Planning and procurement
There were significant problems with the way TfNSW managed the CSELR project between 2011 and 2014. These problems increased the project’s complexity and risk, and reduced the value-for-money outcomes for the State.
To begin with, the established assurance framework provided that TfNSW undertake the assurance reviews of the CSELR project. However, this approach did not provide the independent assurance required for such a major infrastructure project. The CSELR project also departed from the planning process in the State’s Major Projects Assurance Framework. This meant TfNSW skipped two mandatory gateway reviews that could have forced it to resolve deficiencies in the project’s governance arrangements and economic appraisal.
These problems were similar to those we found in our previous reports on WestConnex and Large construction projects. In response, the NSW Government has now strengthened the role of Infrastructure NSW and the Independent Investment Assurance Framework. The CSELR project adopted this framework in mid-2015.
Second, the governance arrangements did not initially reflect the complexity and significance of the project. During planning, there was no dedicated project team and the distinction between commissioning and delivery roles was unclear. TfNSW has now established a dedicated project team and an independent Advisory Board.
Finally, inadequate planning and tight timeframes meant third party agreements and project scope were not finalised before starting the tender process or before signing the main works PPP contract. Scope uncertainty may have increased bid prices and exposed the project to ongoing cost increases.
TfNSW has been progressively finalising the scope of works, and is closely monitoring risks to the timeframe and budget. Some of the project contingency funds have already been applied to changes in the scope of works as a result of TfNSW finalising third party agreements. But contingency funds also need to cover unknown risks that may emerge during construction and delivery. TfNSW considers the current project contingency can cover any further scope changes.
1. For the CSELR project, Transport for NSW should, by December 2016:
a) finalise outstanding design and scope issues
b) ask the project Advisory Board to confirm that controls over the budget and use of contingency funds are consistent with NSW Government decisions and NSW Treasury guidelines
d) ensure the Sydney Light Rail Project Director provides six-monthly briefings to the TfNSW Audit and Risk Committee. 2. For all capital projects, Transport for NSW should comply with the Infrastructure Investor Assurance Framework.
2. For all capital projects, Transport for NSW should comply with the Infrastructure Investor Assurance Framework.
All agencies must show how they achieve value for the public money they spend. For major works such as the CSELR project, agencies should show how they will achieve the objectives and goals of a policy commitment in a cost-effective way. This should be set out in the project’s business case, implementation plans and procurement model.
1.1 Project planning and assurance
There was no independent assurance review of the project business case
TfNSW managed the gateway assurance review of the CSELR project business case. It was within its authority to do so because, between 2011 and 2014, Infrastructure NSW delegated assurance reviews of major transport projects to TfNSW using the TfNSW Investment Gating and Assurance Framework.
Despite this delegation being provided for in the assurance framework at the time, this approach did not provide the independent assurance required for such a major project. Our previous report on Large construction projects in 2015 found TfNSW reviews were robust and comprehensive, but not sufficiently independent.
Independent gateway reviews complement good internal controls. They provide a fresh set of eyes and arm’s length independence not available from even the best internal controls. This is a key principle of the NSW Government’s Major Projects Assurance Framework.
In 2015, the NSW Government strengthened this assurance process. Infrastructure NSW now independently administers the Infrastructure Investor Assurance Framework for capital projects, which outlines a risk-based approach to project assurance. This includes:
- gateway reviews and health checks in line with the project risk profile
- advice to the NSW Government of any risks.
Two mandatory gateway assurance reviews were skipped
In 2011, the NSW Government endorsed the Major Projects Assurance Framework (MPA Framework). This set mandatory independent assurance reviews at all seven stages of a capital project’s life cycle, along with regular reporting to Cabinet.
However, the project-specific planning process the NSW Government approved for the CSELR project departed from this framework. While TfNSW’s process addressed many aspects of the MPA framework, the CSELR project suffered from:
- a lack of independence in assurance reviews
- tight project timelines without justification
- poor assessment of costs and benefits
- weak project governance in the planning stages.
We found many of the same problems in our reports on WestConnex, Large construction projects and the Albert ‘Tibby’ Cotter Walkway.
Normally, after an initial project justification gateway assurance review, agencies must complete two steps before progressing to a final business case:
- a preliminary business case
- a strategic assessment gateway review.
The NSW Government did not require TfNSW to complete these steps for the CSELR project.
Instead, TfNSW set out to deliver the NSW Government policy to build a light rail network in the CBD. In August 2011, it started developing the Sydney Light Rail Strategic Plan (Strategic Plan) to assess light rail routes in the CBD from Circular Quay to Central railway station, and on to Randwick, Kingsford and the University of Sydney.
In December 2012, the NSW Government announced its preferred light rail route from six options in the Strategic Plan. TfNSW then based its final business case on the selected route.
TfNSW considers that the Strategic Plan addressed NSW Treasury requirements for a preliminary business case. And this plan did outline various route options and a bus alternative. However, it did not include elements that are normally part of a preliminary business case, such as the:
- affordability of the project and the justification for Budget priority
- rigorous analysis of options, costs, benefits, risks and sustainability issues
- proposed governance framework.
The two missed assurance reviews would have prompted TfNSW to resolve deficiencies in the economic appraisal and governance arrangements that the final business case retained.
Tight timeframes increased project risk
TfNSW planned the CSELR project to meet the NSW Government’s commitment that work would start in 2014 and include a public private partnership (PPP) arrangement. TfNSW met these timeframes to award two main contracts, including the PPP.
However, meeting such a tight timeframe meant an inadequate business case, poor governance in the planning stage, and uncertain scope during tendering. These combined to increase the project’s complexity and risk, and reduce value for money for the State.
Assessment of costs and benefits was known to be poor but not addressed
Next, the economic appraisal with the final business case underestimated costs and overestimated benefits. It did not adequately account for significant costs and disadvantages. This improved the reported benefit-to-cost ratio, which was assessed as 2.4 (excluding wider economic benefits) when the NSW Government announced the project. Exhibit 3 sets out key issues with the appraisal.
Exhibit 3: Issues with the CSELR project economic appraisal
Source: Audit Office research 2016.
TfNSW was aware of some of these issues, but did not adjust the final business case enough before submitting it for approval. After it was approved, TfNSW still needed to address outstanding issues such as refining the economic appraisal and governance structure, and finalising third party agreements.
Some of these issues have since increased the project cost significantly. We discuss this further in Chapter 2.
TfNSW strengthened governance after final business case was approved
Further, the initial project governance framework for the CSELR project was not effective. There was no dedicated project team or clear distinction between commissioning, assurance and delivery roles.
During the project planning stage, decision-making accountability for the project rested with a Deputy Director‑General of TfNSW. He was the nominated CSELR Project Sponsor but was also responsible for delivering many other major transport projects.
While this is not unusual for TfNSW projects in the concept stage, there is an inherent conflict in not separating the commissioning (Project Sponsor) and delivery (Project Director) roles for a project of this scale.
The Sponsor’s role is to define the outcomes and quality required to meet broader government and organisation objectives. The Sponsor also monitors progress and evaluates results. If the commissioning and delivery roles are combined, there may be a temptation for the Sponsor/Director to define outcomes and quality according to what s/he is able to deliver, rather than what will meet objectives. The assessment of progress and results is similarly conflicted.
We note TfNSW has since addressed this concern. After the NSW Government approved the final business case, TfNSW consulted the Department of Premier and Cabinet, NSW Treasury and Infrastructure NSW on an appropriate governance model. An independent Advisory Board was appointed from January 2014. The Board provides assurance and strategic oversight of the procurement and delivery stages so there is an independent, critical review of how TfNSW is managing the CSELR project.
In April 2015, the Secretary of TfNSW also strengthened project management. He set up a dedicated delivery office for the project, with the Project Director reporting directly to him. Ideally, this governance structure should have been in place during planning and business case development.
We also note that in mid-2015, Infrastructure NSW strengthened its assurance role to:
- conduct regular health checks of the project
- administer gateway reviews at key stages in line with the Infrastructure Investor Assurance Framework
- report monthly to the NSW Government.
While these processes all represent good practice, putting them in place before the business case was approved would have strengthened planning and helped to maximise value for money.
1.2 Project risks
Tight timing in the planning process also increased the risks of scope creep. TfNSW signed the two main contracts before it finalised key third party agreements, so the project design and scope of works were not yet complete.
Scope uncertainty increased the risks to obtaining best value pricing
In the final business case, TfNSW noted third party agreements needed to be in place to mitigate the risk of scope creep. Before the two main contracts were awarded, internal and external reviews repeatedly drew attention to the need to finalise agreements with stakeholders such as utility providers and local councils to complete the project’s design and scope of works.
However, we found TfNSW did not finalise agreements with 12 key stakeholders before starting the tender process for the main works PPP contract on 7 March 2014. It told bidders it would finalise the agreements and update the scope of works halfway through the request for proposal (RFP) consultation period. This had not happened when tenders closed on 11 July 2014.
It also signed the contract for the early works package, noting the risk of scope creep and the cost increases that might occur due to issues that had not been finalised.
TfNSW was responsible for resolving any scope uncertainty to get the best outcome from a competitive RFP process. It did not meet this responsibility. As a result, we cannot assess the extent to which bidders may have included risk-pricing in their bids to compensate for this uncertainty.
Put simply, bid prices may have been higher than if the scope of works had been finalised before tendering.
Ongoing changes in scope continue to increase risk and complexity
TfNSW advised it has been finalising third party agreements since the two main contracts were signed. At the time of writing this report, only one agreement remains unresolved. However, negotiations are continuing with the selected contractors over scope changes. This has increased complexity as TfNSW and the contractors have had to divert resources to requesting, assessing and negotiating design and scope of works changes in contract modifications.
TfNSW renegotiated aspects of the early works managing contract as changes in scope occurred. But because the contract had been awarded, there was reduced pressure on the contractor to offer best value prices for new or revised elements.
TfNSW acknowledges that after appointing the contractor, there was no longer any competitive tension to achieve value for money.
The PPP contract allows both TfNSW and the contractor to propose modifications as the project progresses. Where TfNSW does so, it assesses the value for money of the contractor’s estimated price before confirming the change. The PPP contract also allows the contractor to claim reimbursement if the cost to deliver a contract item exceeds its estimate due to circumstances outside its control.
For some of the agreements finalised after the PPP contract was awarded, TfNSW is waiting for the PPP contractor to assess the impacts of requests to modify the design and scope of works. The large number of unresolved scope issues is unusual for a PPP contract.
TfNSW and Advisory Board are monitoring contingency
TfNSW advised us that:
- project contingency funds will cover any contract variations
- such funds have been ‘ring-fenced’ within the project budget
- scope changes will be managed within the existing contingency allowance.
TfNSW closely monitors risks that could affect the timeframe and budget, with independent oversight by the Advisory Board and Infrastructure NSW. TfNSW presents a register of modifications and contractor claims to the Advisory Board. It also presents a quantified risk and contingency management update to provide evidence there is enough contingency to manage scope risk. In February 2016, TfNSW reported to the Advisory Board that it had received 25 modification requests.
At the February 2016 Advisory Board meeting, the TfNSW contingency update reported it is 91 per cent confident it will deliver the CSELR project within the cost estimate of $2.1 billion. TfNSW advised that it now knows the design and scope changes from the finalised third party agreements, and has applied some of the project contingency funds to these changes.
Of course, contingency funds also need to cover unknown risks that may emerge during construction and delivery.