Overview
This report found the Department of Family and Community Services (FACS) needs to do more to demonstrate it is effectively and efficiently contracting NGOs to deliver community services in the Permanency Support Program (a component of out-of-home-care services) and Specialist Homelessness Services. It notes that FACS is moving to an outcomes-based commissioning model and recommends this be escalated consistent with government policy.
Executive summary
Government agencies, such as the Department of Family and Community Services (FACS), are increasingly contracting non-government organisations (NGOs) to deliver human services in New South Wales. In doing so, agencies are responsible for ensuring these services are achieving expected outcomes. Since the introduction of the Commissioning and Contestability Policy in 2016, all NSW Government agencies are expected to include plans for customer and community outcomes and look for ways to use contestability to raise standards.
Two of the areas receiving the greatest funding from FACS are the Permanency Support Program and Specialist Homelessness Services. In the financial year 2017–18, nearly 500 organisations received $784 million for out-of-home care programs, including the Permanency Support Program. Across New South Wales, specialist homelessness providers assist more than 54,000 people each year and in the financial year 2017–18, 145 organisations received $243 million for providing short term accommodation and homelessness support, including Specialist Homelessness Services.
In the financial year 2017–18, FACS entered into 230 contracts for out-of-home care, of which 49 were for the Permanency Support Program, representing $322 million. FACS also entered into 157 contracts for the provision of Specialist Homelessness Services which totalled $170 million. We reviewed the Permanency Support Program and Specialist Homelessness Services for this audit.
This audit assessed how effectively and efficiently FACS contracts NGOs to deliver community services. The audit could not assess how NGOs used the funds they received from FACS as the Audit Office does not have a mandate that could provide direct assurance that NGOs are using government funds effectively.
Conclusion
FACS cannot demonstrate it is effectively and efficiently contracting NGOs to deliver community services because it does not always use open tenders to test the market when contracting NGOs, and does not collect adequate performance data to ensure safe and quality services are being provided. While there are some valid reasons for using restricted tenders, it means that new service providers are excluded from consideration - limiting contestability. In the service delivery areas we assessed, FACS does not measure client outcomes as it has not yet moved to outcomes-based contracts. FACS' procurement approach sometimes restricts the selection of NGOs for the Permanency Support Program and Specialist Homelessness Services
FACS has a procurement policy and plan which it follows when contracting NGOs for the provision of human services. This includes the option to use restricted tenders, which FACS sometimes uses rather than opening the process to the market. The use of restricted tenders is consistent with its procurement plan where there is a limited number of possible providers and the services are highly specialised. However, this approach perpetuates existing arrangements and makes it very difficult for new service providers to enter the market. The recontracting of existing providers means FACS may miss the opportunity to benchmark existing providers against the whole market. FACS does not effectively use client data to monitor the performance of NGOs funded under the Permanency Support Program and Specialist Homelessness Services
FACS' contract management staff monitor individual NGO performance including safety, quality of services and compliance with contract requirements. Although FACS does provide training materials on its intranet, FACS does not provide these staff with sufficient training, support or guidance to monitor NGO performance efficiently or effectively. FACS also requires NGOs to self-report their financial performance and contract compliance annually. FACS verifies the accuracy of the financial data but conducts limited validation of client data reported by NGOs to verify its accuracy. Instead, FACS relies on contract management staff to identify errors or inaccurate reporting by NGOs. FACS' ongoing monitoring of the performance of providers under the Permanency Support Program is particularly limited due to problems with timely data collection at the program level. This reduces FACS' ability to monitor and analyse NGO performance at the program level as it does not have access to ongoing performance data for monitoring service quality. In the Specialist Homelessness Services program, FACS and NGOs both provide the data required for the National Minimum Data Set on homelessness and provide it to the Australian Institute of Health and Welfare, as they are required to do. However, this data is not used for NGO performance monitoring or management. FACS does not yet track outcomes for clients of NGOs
FACS began to develop an approach to outcomes-based contracting in 2015. Despite this, none of the contracts we reviewed are using outcomes as a measure of success. Currently, NGOs are required to demonstrate their performance is consistent with the measures stipulated in their contracts as part of an annual check of their contract compliance and financial accounts. NGOs report against activity-based measures (Key Performance Indicators) and not outcomes. FACS advises that the transition to outcomes-based contracting will be made with the new rounds of funding which will take place in 2020–2021 for Specialist Homelessness Services and 2023 for the Permanency Support Program. Once these contracts are in place, FACS can transition NGOs to outcomes based reporting. Incomplete data limits FACS' effectiveness in continuous improvement for the Permanency Support Program and Specialist Homelessness Services
FACS has policies and procedures in place to learn from past experiences and use this to inform future contracting decisions. However, FACS has limited client data related to the Permanency Support Program which restricts the amount of continuous improvement it can undertake. In the Specialist Homelessness Support Program data is collected to inform routine contract management discussions with service providers but FACS is not using this data for continuous improvement. |
1. Key findings
FACS has recently restricted the selection of NGOs to provide services for the Permanency Support Program and Specialist Homelessness Services
FACS has a procurement policy framework and a procurement plan, instructions for conducting a market analysis, selecting the sourcing method, and detailed lists of the documentation required throughout the process. Although some new providers were contracted in 2012 and 2014, no new service providers have been contracted since then as FACS has used restricted tender processes rather than opening the process to the market. The use of restricted tenders is consistent with FACS' procurement plan and permissible in situations where one or more of the following apply:
- the nature and range of the goods or services being sought is well defined and known
- the geographic area for the service is small within a region or remote area
- there is only a small number of specialised suppliers in the market capable of delivering the contract
- the capacity of existing suppliers to cope with demand and growth is known
- the relative past performance of existing supplier(s) under similar contracts is determined to be satisfactory.
However, restricting tenders makes it very difficult for potential new service providers to enter the market, perpetuating the selection of existing service providers. The recontracting of existing providers also means that FACS may miss the opportunity to benchmark existing providers against the whole market.
Processes to manage NGO performance and quality are ineffective to ensure delivery of the objectives of the Permanency Support Program and Specialist Homelessness Services
For the program areas we examined, FACS has few process in place to ensure that clients receive safe, quality services and relies on its contract management staff to monitor day to day NGO performance. Although FACS does provide some training material on its intranet, limited training has been provided to FACS contract management staff on how to perform this monitoring role. There is also limited guidance available to staff and confusion about roles and responsibilities between regional and Central offices.
For example, FACS is not adequately monitoring the Permanency Support Program due to known problems with the ChildStory system. This has resulted in low NGO compliance with data requirements as well as inaccurate and incomplete client data. FACS has demographic data on clients in the Permanency Support Program but has incomplete or missing data on items such as case plans for children and young people in care. This limits FACS' ability to monitor and analyse NGO performance and as a result, FACS' contract staff are expected to monitor day to day NGO performance and advise NGOs when improvements need to be made.
In Specialist Homelessness Services, providers supply data directly to the Australian Institute of Health and Welfare (AIHW), as does FACS. NGOs should also be providing all client data to FACS but do not always do so and, when they do, FACS is not making use of this data for NGO performance management or improvement.
Communication within FACS relating to the management of NGO performance issues is inconsistent, which raises the risk that different contract managers may respond differently to NGO performance issues. For example, when performance issues with an NGO are identified, many are dealt with at the local level by FACS contract management staff. Problems are not always reported to FACS Central Office. FACS needs to ensure consistency both in the reporting of issues to the Central Office and in the response to issues across regions and teams.
FACS relies on external accreditation to ensure NGO minimum standards
By law, all out-of-home care service providers must be accredited by the Office of the Children's Guardian to be eligible to operate in New South Wales, including providers in the Permanency Support Program.
Accreditation is awarded to providers that demonstrate that their services meet the minimum acceptable requirements for out-of-home care and adoption providers. The Office of the Children's Guardian monitors all accredited service providers to ensure they are compliant with these standards. In using accredited service providers, FACS is only working with providers certified by the Office of the Children's Guardian as demonstrating at least the minimum acceptable standards of governance in providing services for children and young people.
FACS is currently introducing accreditation for Special Homelessness Services with compliance required by 30 June 2023. The accreditation for Special Homelessness Services is the Australian Service Excellence Standards and all providers must achieve at least the certificate level of accreditation which encourages organisations to introduce plans for implementing and monitoring continuous improvement.
FACS is unable to evaluate outcomes for clients of NGOs
NGOs are required to complete a quarterly report of their activity and these reports are consolidated to form part of the annual accountability process. This process requires NGOs to self-report their financial accounts and also to demonstrate that their performance has met their contractual obligations. However, current contracts only require NGOs to report on activity-based performance metrics rather than client outcomes.
The transition to outcomes-based contracting will be made with the new rounds of funding which will take place in 2020–2021 for Special Homelessness Services and 2023 for the Permanency Support Program. Thereafter, NGOs will be contractually required to report client outcomes. The transition to outcomes-based contracting will also make FACS contracting processes more consistent with the NSW Government Human Services Outcome Framework and the Commissioning and Contestability Policy.
Incomplete data limits FACS' effectiveness in continuous improvement for the Permanency Support Program and Specialist Homelessness Services
FACS has policies and procedures in place to learn from past experiences and use lessons to inform future contracting decisions. However, in the Permanency Support Program, the introduction of ChildStory has resulted in issues including low system uptake in the NGOs and inaccurate or missing data. This limits the amount of continuous improvement and performance evaluation FACS can undertake.
In Specialist Homelessness Services, continuous improvement is part of the Monitoring and Evaluation Strategy. Reducing youth homelessness is also a Premier's Priority and data representing progress toward this goal is gathered at a system level.
FACS has introduced a process for conducting maturity assessments at the program level to assess how well the program is achieving its intended commissioning objectives and to develop and embed better practice. In late 2018, FACS Commissioning Excellence branch developed and piloted a maturity indicators framework to identify strengths and weakness in each program and determine where FACS should focus its efforts.
2. Recommendations
This audit was conducted with the Department of Family and Community Services as the auditee. On 2 April 2019 the NSW Government announced it will combine FACS cluster with the Department of Justice to form the Department of Family and Community Services and Justice, effective from 1 July 2019. The recommendations below are directed to the Department of Family and Community Services and Justice, which will assume responsibility for the Permanency Support Program and Specialist Homelessness Services once these changes have been introduced.
By June 2020, the Department of Family and Community Services should:
- Implement measures to conduct full program level market analysis of services to be provided by NGOs to:
- identify potential new service providers
- ensure that benchmarking of NGO performance and quality is conducted against the market.
- Clarify roles and responsibilities and provide clear guidance for FACS contract management staff to ensure effective and consistent management of NGO performance and quality.
- Develop a process for reporting all NGO performance and quality issues to the Central Office to inform performance management and contracting management decisions.
- Escalate the move to outcomes-based contracts for NGO contracted services by:
- reviewing which services delivered by NGOs can move to outcomes-based contracts and the timeline for doing so
- setting clear and measurable targets for the number of programs moved to outcomes-based contracts
- publicly reporting progress against these targets
- continuing to build capability within FACS to manage outcomes-based contracts.
- Improve the accuracy and completeness of client and NGO performance data and use it to:
- monitor the performance and quality of NGO services
- provide performance feedback to NGOs on their services
- publicly report the performance of NGO services.
1. Introduction
1.1 Background
The Department of Family and Community Services
The Department of Family and Community Services (FACS) works with children, adults, families and communities to improve lives and help people realise their potential. FACS is also the agency within the NSW Government with statutory responsibility for protecting children and young people from risk of serious harm.
FACS is composed of seven Districts which are responsible for the delivery of services across the state and align with the NSW Local Health Districts.1 FACS contracts NGOs to deliver many of its services but retains stewardship of those services and has the declared intention of improving both outcomes and experiences for its clients. In the financial year 2017–18, FACS spent $6.4 billion on service provision. More than $1.9 billion was spent on 3,293 payments to NGOs.2
FACS provides human services across many areas but this audit focussed on these two areas of service delivery:
- The Permanency Support Program (PSP) which is part of out-of-home care (OOHC).
- Specialist Homelessness Services (SHS).
The Permanency Support Program
Between 2010 and 2016, the number of children in out-of-home care in New South Wales rose by more than 30 per cent to just over 15,000 and the average time that a child remained in care was 12.6 years (as at 1 July 2014). In 2015–16 the Independent Review of Out-of-Home Care (also known as the Tune report) was completed which noted that most out of home care focussed on crisis response rather than early intervention and prevention.
The recommendations of this review focused on:
- reducing the numbers of children entering care
- helping children return to safe homes from care
- creating permanency through guardianship and open adoption where children are unable to safely return home
- improving the medium and long-term outcomes for children in care and those leaving care
- applying the commissioning approach to outcomes and accountability across both FACS and non-government service provision.
In response to this review, FACS introduced the Permanency Support Program on 1 October 2017 with full implementation of the program completed by 1 July 2018. The Permanency Support Program is designed to provide tailored services to vulnerable children and to provide them with permanency, which gives children a better chance of living successfully and independently as adults.
Specialist Homelessness Services
The Specialist Homelessness Service program is the primary NSW Government response to homelessness and supports people from a wide variety of different situations who are experiencing or who are at risk of homelessness.
In the year 2017–18, the Australian Institute of Health and Welfare (AIHW) reported that there were 71,628 people who were homeless or at risk of homelessness in New South Wales. This number has been growing at an average of 5.4 per cent annually over the period 2011–12 to 2017–18, which is higher than the national growth rate over the same period.
The NSW Government Human Services Outcome Framework
The NSW Government has introduced a Human Services Outcome Framework (the Framework) to ensure that commissioned services better meet the needs of people and to assist agencies in commissioning community services.
The Framework attempts to achieve this goal by ensuring the development of outcomes and suitable performance measures for all commissioned services, although its use is not mandatory. The Framework sets out the population outcomes that the NSW Government is seeking to achieve in the delivery of human services. This will assist the government to meet the needs of the people of New South Wales by:
- building a common understanding of the outcomes for people that are priorities across agencies and NGOs
- supporting agencies and NGOs to adopt an outcomes-focused approach in human services planning and commissioning
- promoting consistency in the measurement and evaluation of outcomes and activities
- fostering innovation, learning and improvement
- encouraging agencies and NGOs to work together to achieve outcomes.
The Framework includes seven domains which identify the intended outcomes for the population of New South Wales and reflect the factors that impact these outcomes for vulnerable people. With the exception of Premier's and State Priorities, the Framework does not specify how these outcomes are to be measured and expects agencies to develop suitable indicators for their own contracts and programs. Exhibit 1 illustrates the outcome domains in the Framework.
Due to FACS' increased reliance on NGOs for service provision, the Framework impacts strongly on this agency. FACS began to develop an outcomes-based approach to contracting in 2015 and has designed an approach which focusses on helping individuals develop the capability to realise the outcomes they have chosen for themselves.
The Commissioning and Contestability Policy
In 2016, the NSW Government introduced a new Commissioning and Contestability Policy (TPP16-05) (the Policy) which applies to all NSW Government agencies. The Policy is designed to provide a ‘whole-of-government strategic approach to the delivery of quality services to sustainably provide improved outcomes for New South Wales citizens’.
The Policy is to be used:
- in developing new service delivery policies and proposals, or when service delivery policies undergo significant review
- when undertaking portfolio or service delivery reviews
- where commissioning and contesting services impact annual and mid-year budget submissions
- when undertaking strategic planning and whole-of-agency and/or cross-agency organisational redesign
- prior to executing major contracts that bind government for long contract periods, to ensure they are as contestable as possible
- in determining service continuity arrangements prior to the end of major contracts, particularly when the contract and industry is affected by innovation.
Under this policy, commissioning is defined as an ‘approach to considering the outcomes that need to be achieved, and designing, implementing and managing a system to deliver these outcomes in the most effective way’. In contrast, contestability is defined as the ‘process of evaluating and benchmarking services against credible alternatives and/or market testing in order to drive productivity, learning and improvement’. NSW Government agencies are required to ensure that all of their service delivery policies are in line with this Policy.
2 This expenditure includes around $1.0 million in pass-through grants made to the National Disability Insurance Agency.
2. Contracting for human services
2.1 Procurement of human services
FACS has a procurement approach which restricts the selection of NGOs to provide community services
FACS has a procurement policy framework which details the legal, administrative and governance framework for procurement. The aims of this framework include:
- providing a clear process to support impartial and consistent decision making
- streamlining funding processes and ensuring they are commensurate with risk and the funding available
- providing increased information to provide transparency, create awareness and maximise competition.
In addition, FACS has a detailed structure for planning procurement including instructions for conducting a market analysis, selecting the sort of tender or negotiation approach to be used, and detailed lists of the documentation required throughout the process.
FACS has an exemption to Human Service Agreement contracts in out-of-home care
Under Procurement Board Direction 2017-04, from August 2017, all NSW Government agencies are required to use the prescribed NSW Human Services Agreement (HSA) when procuring human services from NGOs. Agencies are required to switch to this form of agreement as existing contracts expire. This contract is designed to:
- make it easier for NGOs to do business with NSW Government agencies by streamlining contracting arrangements
- promote consistency across NSW Government
- be simple, user friendly and fit for purpose
- reflect the partnership needed by NGOs and agencies to deliver better human services outcomes.
Transitioning to using the Human Services Agreement required FACS to transition to performance-based contracts as the HSA Schedule specifies both performance outcomes and the measures that the NGO must report. However, in August 2017, the NSW Procurement Board granted FACS an exemption from this Direction for all out-of-home care contracts, including the Permanency Support Program, that were due to be renewed in 2017 and 2018. The current contracts are due to expire in 2023.
The exemption was granted on the grounds that FACS had been in the process of recontracting out-of-home care for some months when the Procurement Board Direction was issued and there was insufficient time to switch to the new contracts before the current contacts expired. FACS contended that enforcing the change at such a late stage in the recontracting process could have resulted in disruption to services which was considered an unacceptable risk.
FACS sometimes uses restricted tenders in the Permanency Support Program, limiting contestability
FACS has a program specific procurement process for the Permanency Support Program, which is consistent with its procurement policy, and includes:
- market analysis
- the choice of sourcing method
- evaluation plans for both expressions of interest and responses to a request for tender
- timelines for the procurement process.
In 2012, FACS used an open process to invite tenders from new out-of-home care providers. The aim of the tender was 'to establish the pre-qualified lists and proposed growth placements in each Community Services region'. Thirty new providers entered the market at that time.
In contrast, in 2017, to limit disruption to service provision, FACS restricted its tender process for the Permanency Support Program to existing providers. The exception to this was Intensive Therapeutic Care (formerly known as Residential Care), which is a specialised and intensive support system for children and young people with complex needs. In this case, a two-stage procurement process was used. In stage 1, Expressions of Interest were invited from the market, and a list of preferred providers drawn up. In stage 2, the preferred providers were then invited to submit a tender proposal.
FACS permits the use of selective tenders under its procurement framework in certain specific circumstances. This includes situations when:
- the nature and range of the goods or services being sought is well defined and known
- the geographic area for the service is small within a region or remote area
- there is only a small number of specialised suppliers in the market capable of delivering the contract
- the capacity of existing suppliers to cope with demand and growth is known
- the relative past performance of existing supplier(s) under similar contracts is determined to be satisfactory.
In the Permanency Support Program, potential providers are required by legislation to be accredited with the Office of the Children's Guardian, which further limits the pool of potential service providers. The use of restricted tenders makes it very difficult for new service providers to enter the market. The recontracting of existing providers also means that FACS may miss the opportunity to benchmark existing providers against the whole market. This limits contestability as benchmarking services against credible alternatives is an element of contestability, as defined by the Policy.
FACS sometimes uses recontracting for Specialist Homelessness Services, limiting contestability
In 2014, the Going Home Staying Home reform agenda was introduced. This focussed on longer term results for homeless people and those at risk of homelessness. In this round of funding, FACS employed a two-stage process involving a prequalification scheme followed by a tender. The prequalification stage was open to all eligible market participants, but the tender was restricted to organisations that met the criteria for prequalification.
In 2016, to be consistent with Cabinet direction that there be 'minimal disruption', FACS restricted the tender process for Specialist Homelessness Services to existing suppliers. FACS has advised that this was done to minimise further disruption to the sector as the previous financing round
(in 2014) was considered particularly difficult for NGOs. In 2014, a compressed timeframe and the introduction of Joint Working Agreements made it difficult to service providers to put together effective tenders.3
The next round of funding for Specialist Homelessness Services, due in 2020–2021, will not involve a procurement process as the decision has been made to recommission existing service providers to minimise disruption to services. Only those providers with 'significant performance issues' resulting in the use of a Performance Improvement Plan (PIP) will be excluded from this process.
A project plan for recommissioning Specialist Homelessness Services also exists for the upcoming round of financing and the transition to outcomes- based contracting. This plan includes:
- consultation with FACS Districts and providers on outcomes and indicators to be used
- the pilot program currently underway and the evaluation of the results
- updating and circulating Specialist Homelessness Services program guidelines
- negotiating and signing new contracts for Specialist Homelessness Services service provision.
The recontracting of existing providers means that FACS may miss the opportunity to benchmark existing providers against the whole market. It also makes it very difficult for new service providers to enter the market.
FACS also uses alternative funding to fund NGOs that did not receive funding under Specialist Homelessness Services
Since 2014, a further group of small and medium sized NGOs have been funded by the Service Support Fund to provide homelessness services. These NGOs did not receive funding in the 2014 funding round for the Specialist Homelessness Services resulting in 'significant contentious public reaction'4 which led to the introduction of the Service Support Fund. The Service Support Fund is intended to 'deliver new programs that complement specialist homelessness services and the wider approach to reducing homelessness'.5 FACS has also advised that these organisations were funded through the Service Support Fund to mitigate the impact that losing Special Homelessness Service funding would have on these organisations.
To be eligible for this funding, NGOs had to meet criteria including:
- having been previously funded through the Special Homelessness Service or the National Partnership Agreement on Homelessness
- not receiving any Special Homelessness Service funding after November 2014
- being a small to medium sized NGO
- being predominantly a homelessness organisation that has principally relied on the Special Homelessness Services or the National Partnership Agreement on Homelessness.
These NGOs were then required to provide a project plan to demonstrate that the service they offered filled a gap in the existing homelessness provisions. The initial funding for these service providers was intended to be for just 18 months but has been extended to 2020 alongside the Specialist Homelessness Services funded services.
In the next contracting round, in 2020, FACS intends to continue this funding and recontract this group of NGOs provided they do not have significant performance problems, as with the Specialist Homelessness Service providers above. In the 2020 recommissioning of homelessness services, FACS intends to align the service delivery arrangements of Service Support Fund providers with other providers funded through the broader homelessness program.
However, as in the Permanency Support Program, the use of restricted tenders and inviting select NGOs to submit project plans makes it very difficult for new service providers to enter the market. Once again, the recontracting of existing providers means that FACS may miss the opportunity to benchmark existing providers against the whole market, which limits contestability.
4 Documentation on Item 7 'SHS Program Budget and Recommissioning' for the FACS Housing and Homelessness Strategy Steering Committee, 2 November 2018, page 2.
5 FACS web page 'Our homelessness services' as at 27 March 2019, https://www.facs.nsw.gov.au/providers/funded/programs/homelessness/specialist-services/our-programs.
3. Monitoring and performance reporting
3.1 Accreditation of service providers
FACS relies on external accreditation to ensure minimum standards are met for out-of-home care
By law, all Permanency Support Program service providers must be accredited with the Office of the Children's Guardian (OCG) to be eligible to operate in New South Wales. To be accredited, a service provider must demonstrate that their organisation complies with the NSW Child Safe Standards for Permanent Care (the Standards). These standards establish the minimum requirements for out-of-home care and adoption providers. They are based on the statutory responsibilities set out in the Children and Young Persons (Care and Protection) Act 1998, the Adoption Act 2000 and other relevant regulations.
According to the OCG, the 'purpose of accreditation is to ensure that the rights of children and young people are upheld, and that children and young people will receive quality services, regardless of where they are placed. The accreditation scheme also requires agencies to continually strive to improve the care they provide to children and young people'.
The OCG monitors all accredited service providers to ensure they are compliant with the Standards. If there are any concerns about compliance, the service provider can be placed onto a Monitoring Program. If this happens, then the OCG can:
- place the service provider on an action plan which includes timeframes for resolving problems
- visit service providers at least once every 12 months
- suspend accreditation if the service provider fails to meet the Standards after being given the opportunity to rectify their situation.
FACS holds regular meetings with the OCG where matters of concern can be raised on either side and plans made to rectify any shortfalls in compliance with the Standards. Information about potential issues can be passed from FACS contract management staff to the OCG either directly or via these meetings. The OCG then investigates any issues that are of concern.
FACS is only working with providers certified by the OCG as demonstrating at least the minimum acceptable standards of governance in providing services for children and young people.
Accreditation is being introduced for Specialist Homelessness Services
In contrast to the Permanency Support Program, accreditation has not previously been required in Specialist Homelessness Services. FACS is introducing accreditation into this program and all providers must begin the process of becoming accredited by 2020. Any organisation which is not accredited by 30 June 2023 will no longer be eligible for funding under the Specialist Homelessness Services program or any other FACS homelessness program.
The accreditation for Specialist Homelessness Services is the Australian Service Excellence Standards (ASES). All providers must achieve at least the certificate level of accreditation, which encourages organisations to introduce plans for implementing and monitoring continuous improvement.
ASES was selected as the form of accreditation as it most closely matches the Specialist Homelessness Services Quality Assurance System standards that were already in place. As with the Permanency Support Program, FACS is only working with providers certified by the Australian Service Excellence Standards as demonstrating at least the minimum acceptable standards of governance in providing homelessness services.
3.2 Quality, safety and performance monitoring
FACS is not collecting timely or quality performance data from NGOs funded through the Permanency Support Program and Specialist Homelessness Services
All the NGOs funded under the programs we reviewed are required to report on some performance measures, but the data they report is of variable quality and completeness. FACS has issues with its systems which limit the amount of data that can be reported and the analysis that can be carried out. This makes it difficult to determine how well service providers are performing and makes it difficult for FACS to demonstrate that it is efficiently managing the delivery of human services.
Until November 2017, FACS produced quarterly dashboards for its Permanency Support Program providers which contained data about the amount of funding received from FACS and also included the number of:
- placements funded by FACS
- children exiting care
- children receiving a risk of serious harm (ROSH) report
- cases managed by the agency
- reportable conduct notifications6
- formal and complex complaints.
This data was available to FACS contract managers, the Directors of Statewide Contracting and the District Directors of Commissioning and Planning to facilitate performance reviews.
FACS stopped producing key performance dashboards for NGOs in November 2017 in anticipation of the introduction of the ChildStory platform at the end of 2017, and the full implementation of the Permanency Support Program in July 2018.
ChildStory is an integrated information technology system that allows the development of a network, involving family, carers, caseworkers and service providers, to work together to provide safety and wellbeing for a child. ChildStory was implemented in November 2017. At the current time, ChildStory is available to:
- FACS staff
- child protection practitioners in Police, Health and Education
- non-government organisations
- other government agencies
- mandatory reporters.
However, since the introduction of ChildStory, there have been reported problems including low user uptake in the NGOs and issues with inaccurate or missing client data according to a management review commissioned by FACS. This means that FACS has demographic data on clients in the Permanency Support Program but has incomplete or missing data on items such as case plans for children and young people in care. This indicates that reporting in the Permanency Support Program is currently inadequate, with limited performance data being provided for NGOs and difficulty for FACS to accurately monitor NGO performance. These issues mean that FACS is unable to accurately monitor NGOs performance or undertake performance reviews. FACS has plans in place to remediate these issues and states that a new dashboard will be available to NGOs in August 2019.
In the Specialist Homelessness Services, NGOs are required to provide data to the Australian Institute of Health and Welfare and are expected to provide the same data to FACS. Reducing youth homelessness is a Premier's Priority and system level data representing progress toward this goal is collated at the District level by FACS and reported on a monthly basis.
Limited checks are performed on data collected for the Permanency Support Program and Specialist Homelessness Services
The only checks that are carried out on reported data are based on internal validity. The accuracy of client data reported by NGOs is not confirmed. Instead, FACS contract staff are expected to identify errors in data during the annual accountability process and ensure that these errors are corrected at that time.
The annual accountability process also requires NGOs to submit their financial statements and demonstrate that they have complied with the performance requirements set out in their contract with FACS. The annual accountability process has four objectives which are to:
- ensure government funds are being used as agreed in contracts
- identify any issues affecting a service provider's ability to maintain stable, uninterrupted delivery of services
- determine whether a service provider continues to have capacity and capability to deliver better outcomes for FACS clients
- reveal trends and common issues across programs and Districts.
As part of this process, FACS contract managers are required to check NGO submissions to ensure all the necessary information has been reported. FACS verifies the accuracy of the financial data against the NGO's audited financial statements. However, FACS conducts limited validation of performance and client data reported by NGOs to verify its accuracy. Any errors that are identified at this point are corrected and then the contract managers complete the annual performance and risk assessment for each NGO.
FACS contract staff are also expected to monitor day to day NGO performance, including safety, quality of services and compliance with contract requirements, and advise NGOs on when improvements need to be made and how to make them. In the Permanency Support Program, NGO representatives are contractually required to have monthly meetings with a FACS representative to review the service provider's performance, but no such requirement exists in Specialist Homelessness Services contracts.
FACS provides some training for its contract management staff and more is planned
In order to assist its contract management staff in their interactions with NGOs, FACS has provided some training, and has placed training materials on its intranet. To date this includes:
- an overview of the Funded Contract Management Framework
- guidance on the annual accountability process
- financial training for contract managers.
The introduction of financial training was in response to comments by both FACS staff and NGOs in questionnaires about the 2016–17 financial year. Service providers also requested better training for FACS staff in the annual accountability process. No service providers have been surveyed since this training was introduced so there is no evidence to date of whether it has resolved this issue to the satisfaction of NGOs and other service providers.
FACS has more training planned for the remainder of this financial year including:
- corruption prevention in procurement and contract management 7
- contracting 101
- recordkeeping
- contract negotiation and relationship management training.
FACS is not adequately supporting its contract management staff to manage NGO performance
When performance issues are identified, many are dealt with at the District level by contract officers and managers. There is no central issues register detailing actions for resolution and timeframes. Problems are not always reported to FACS Central Office and there is uncertainty around the roles and responsibilities of Districts and team members.
If problems are particularly serious then they are escalated and the Prudential Oversight unit, based within FACS Central Office, may be involved. At this stage the NGO may be placed on either a Service Development Plan or, more seriously, a Performance Improvement Plan. As these plans may involve abatements or other alterations to the payments a service provider receives from FACS, FACS Central Office is made aware of the issue.
The inconsistencies in reporting raises the risk that different Districts may report different things to the Central Office and may respond differently to NGO performance issues. This also raises the risk that FACS does not have complete information about NGO performance, which limits contestability and could result in the recontracting of poorly performing service providers. FACS needs to ensure consistency both in the reporting of issues to the Central Office and also in the response to issues across Districts and teams.
FACS is developing a Quality Assurance Framework for the Permanency Support Program
FACS is currently developing a Quality Assurance Framework for all clients in out-of-home care, including the Permanency Support Program. The framework was also in development at the time of our previous audit on out-of-home care in 2015. Despite this, it has not yet been implemented. FACS conducted a trial study of its Quality Assurance Framework in 2018 and the results are currently being evaluated. FACS expects to begin implementation of the Quality Assurance Framework in 2019.
The Quality Assurance Framework sets out the outcomes for all children and young people in out-of-home care with respect to:
- Safety: children and young people feel safe, secure and protected from harm
- Permanency: children and young people have residential stability and maintain relationships with family, kin and community
- Wellbeing: academic achievement, mental health, cultural and spiritual identity, social functioning and physical health.
The Quality Assurance Framework may help standardise data collection and reporting and provide a more comprehensive picture of what is happening in a young person's life. The Quality Assurance Framework may assist both FACS and NGOs to identify what goals need to be achieved and should also increase the amount of information available to NGOs about individual children in care which could help in planning the best future for these children.
The Quality Assurance Framework is intended to include data from FACS, NGOs, and other agencies, carers and young people but FACS acknowledges there may be difficulties persuading other agencies to share data at the individual level.
3.3 Transitioning to outcomes-based contracting
FACS is transitioning to outcomes-based contracts
FACS began the development of an outcomes framework in 2015. However, FACS does not yet have any outcomes-based contracts in either the Permanency Support Program or Specialist Homelessness Services. FACS is in the process of transitioning these programs to an outcomes-based approach with the first outcomes-based contracts expected to be in place for Specialist Homelessness Services in July 2020.
The Permanency Support Program was introduced in October 2017 and full implementation of the program was completed by 1 July 2018. In the contracts issued at that time, FACS specifies its intention to enter into outcome-based performance contracts with service providers. The designated outcome for this program is that the child or young person will achieve permanency within two years of entering care.
FACS expects that a complete transition to a commissioning model will take several years and this is listed as a future goal for FACS in both its Customer Service Improvement Plan 2016-2019 and the Strategic Plan 2017-2021.
FACS is yet to track outcomes for clients of NGOs as part of its most commonly used contracting arrangements
Service providers must report both performance measures and outcomes data under these contracts. However, payments made to service providers under these contracts are linked to activity-based measures (Key Performance Indicators) and not outcomes. An example of an activity-based measure for the Permanency Support Program would be that a child has a case plan. In comparison, an outcome would be that a child has a permanent home within two years of entering care. At the time of establishing these contracts, FACS did not have sufficient evidence to determine how the measures introduced in the Permanency Support Program would impact on outcomes for children and young people.
FACS determined that few service providers had the capacity or capability to measure outcomes and so FACS decided that the transition to outcomes-based contracting would be made over time. FACS advises that all Permanency Support Program contracts will be outcomes-based in the next round of financing, which is scheduled for 2023.
One area where outcomes-based contracts have already been employed in New South Wales is in the provision of services funded with Social Benefit Bonds (also known as Social Impact Bonds). Exhibit 2 explains their use and outcomes.
Social Benefit Bonds are a pay-for-performance, or payment for outcomes, contract for government service delivery. A key feature of Social Benefit Bonds is the involvement of private investors. The following figure shows the general features of Social Benefit Bonds. |
Source: Adapted by AO NSW from Gustafsson-Wright, Emily; Sophie Gardiner; and Vidya Putcha, 2015. The potential and limitations of impact bonds: lessons from the first five years of experience worldwide, Brookings Institute, July 2015, page 6.
In a Social Benefit Bond, a private investor provides funds to an intermediary to wholly or partly finance service delivery. The intermediary coordinates and manages service provision with a service provider. These services are designed to make savings for the government when compared either to a benchmark or to the costs of not providing the service at all. The government pays a reward payment to the intermediary from the expected savings and part of this money is then returned to the investors, who receive both their initial investment and the interest it has earned. In 2013, the NSW Government implemented two social benefit bonds involving a small number of participants. These bonds were used to fund early intervention to prevent children and young people entering or re-entering out-of-home care. A third bond, to support young people who have left out-of-home care, is planned for 2020. The Newpin bond The Newpin bond targets family restoration and, as at 30 June 2018, Newpin has recorded 272 restorations to family and achieved a cumulative restoration rate of 63.3 per cent over the first five years of the bond. At the end of the 2017–18 financial year, the returns for Newpin were 13.5 per cent, which exceeds the ten per cent to 12 per cent target return identified in 2015. The Newpin bond will mature in 2020. The Benevolent Society bond The Benevolent Society bond targets family preservation and avoiding entries into out-of-home care. There are two classes of investors for this Benevolent Society bond, capital protected investors and capital exposed investors. Capital exposed investors could potentially lose their up-front payment but will be rewarded with a higher return. The Benevolent Society bond has recorded 19 fewer entries (41 against 60) into out-of-home care compared with a control group over a five-year period. Investors in the Benevolent Society bonds are on track for returns of six per cent and 10.5 per cent as at 30 October 2018, depending on which class of investor they are. The Foyer51 Social Benefit bond A third bond, the Foyer51 bond is expected to start in 2020. It will help assist in addressing youth homelessness by providing support for young adults as they exist the out-of-home care system. Specifically, this bond will provide a safe and affordable place for young people to live while they develop the skills to lead independent lives. |
FACS is piloting outcome indicators for Special Homelessness Services
Special Homelessness Services will be recontracted in 2019 with new contracts starting in 2020. These contracts will contain details of the outcomes that service providers are expected to achieve for their clients and FACS is currently undertaking a pilot study on the best indicators to use.
This pilot study will help develop a system for measuring and reporting on a proposed set of outcome indicators for these contracts and identify any risks arising from the use of these indicators. Following stakeholder consultations, the pilot study is evaluating six indicators reflecting three outcomes; safety, housing and wellbeing. The outcomes are:
- Safety: clients feel safer, clients make progress addressing their safety needs
- Housing: clients make progress addressing their housing needs, clients sustain their tenancy
- Wellbeing: clients have improved personal wellbeing, clients have improved capacity to tackle future challenges.
Data on these outcomes will be collected using information on how clients perceive their situation8 and a survey of how well clients have progressed against the goals set in their case plan. Every client will be given the opportunity to provide this information during their initial assessment, exit interview and during case plan reviews.
Evaluation of the pilot study will overlap with the implementation phase and will be used to determine:
- barriers to implementation
- recommendations for improved data collection
- success factors and improvements to training materials and guidance provided to service providers.
However, payments will not be based on outcomes for the Specialist Homelessness Services program in the 2020 round of funding. FACS has decided that outcomes will not be linked to payments for existing programs until the validity and reliability of the outcome measures has been established and any potential risks fully investigated.
FACS plans to review outcomes measures over the course of the contracts starting in 2020 and use this information to determine suitable baselines for different groups of clients and to understand how best to collect outcomes data. Only then will FACS transfer to an outcomes-based contracting approach.
4. Continuous improvement
4.1 Continuous improvement in contracted human services
FACS has a continuous improvement approach but lacks the data to use it effectively in the Permanency Support Program and Specialist Homelessness Services
As part of the FACS Commissioning Framework, the performance of any program should be measured and evaluated, and the results used to recommend changes or improvements to be made in the future. This process of evaluation and continuous improvement is in both the Treasury Commissioning and Contestability Policy and the accompanying Practice Guide.
However, owing to limitations in the available data, FACS is restricted in the amount of evaluation it can do and the value it can get from this process. NGOs have reported that they would like to receive much more data from FACS if they are to effectively manage their own performance.
In the Specialist Homelessness Services program, continuous improvement is part of the Monitoring and Evaluation Strategy. NGOs' client data in this program is submitted via the SHS Client Information Management System and is measured at three levels. These levels are:
- Population: reductions in the number or rate of homelessness in New South Wales
- Client: beneficial effects for clients
- System: results arising from a redesigned service system and industry.
Service providers supply client data directly to the Australian Institute of Health and Welfare (AIHW) to support national reporting requirements and should be providing the same data to FACS to support internal monitoring and reporting activities. However, this information is not used to assist NGOs with continual improvement.
FACS is conducting maturity reviews to aid program development including 'deep dives'
FACS has established a Commissioning Excellence branch, which undertakes program maturity reviews to assist programs to develop and embed better practice. In 2018, Commissioning Excellence developed and piloted a maturity indicators framework including a number of 'deep-dives' to explore programs that have recently been reformed in depth. Both the Permanency Support Program and Specialist Homelessness Services were the subject of deep dives in November 2018 and these programs were rated as 'establishing' and 'developing', respectively.
The deep dive into the Permanency Support Program found issues and insights including:
- participation needs careful planning and timing to ensure it is a positive experience
- adapting to reform may be difficult for service providers and could impact their financial viability
- District representatives communicate regularly with Head Office to capture learning and resolve issues
- sector capacity is limited, and NGOs need assistance and training to understand how the Permanency Support Program impacts on their operations
- FACS and NGO caseworkers need to have the same information, resources and training to ensure the best results for children and young people
- a lack of accurate data makes it difficult to monitor whether service providers are meeting their contractual obligations
- issues with data may impact on the evaluation of performance measures.
The deep dive into Specialist Homelessness Services found issues and insights including:
- identifying realistic outcome indicators will be challenging
- sharing knowledge both within FACS, with other government agencies and with multiple stakeholders is a challenge
- past reforms have damaged trust with some peak bodies and high turnover of FACS staff makes building relationships difficult
- introducing Australian Service Excellence Standards (ASES) accreditation for all Specialist Homelessness Services providers has been positively received
- FACS Districts play a crucial role in building relationships, contract management and performance monitoring of service providers
- performance data could be better used to provide insights and inform continuous improvement.
FACS learns from past experiences but there is confusion about the best way to disseminate information
FACS has advised that it takes lessons from past experience and incorporates them into current and future decisions. The FACS Commissioning framework stipulates that programs should be regularly evaluated, to improve future performance. FACS also commissions consultants to undertake reviews at both the end and during the life of a program, such as the early review of the 2014 changes to the Specialist Homelessness Services program (known as Going Home Staying Home) which was completed in 2017. The recommendations arising from these reviews then inform the future development of these programs.
In addition to commissioned reports, FACS has multiple committees, working groups and forums which allow it to consult and receive feedback from individuals within FACS, other NSW Government bodies such as the Office of the Children's Guardian, and from stakeholder bodies such as peaks.
However, communication within FACS is inconsistent and some methods of disseminating information to staff are not effective. More than 25 per cent of staff in a Permanency Support Program workshop raised the issue of communication between the Central Office and Districts as something they wanted to have clarified and this was also raised in a questionnaire completed by contract managers.
In 2018, three 'metro' Districts developed their own communications protocol to ensure that stakeholders are kept informed of changes in practice and to promote consistent practices in the Permanency Support Program.9 The need for better communication with NGOs was raised by service providers in an equivalent questionnaire and NGOs have also expressed concerns about the poor communication between the Districts and Central Office which also has an impact on their performance.
Communication with NGOs may be improved with the introduction of a new Contract Management system, which should have the capability to function as a Customer Relationship Management system. Nonetheless, the risk still exists that different Districts may respond differently to NGO performance issues. FACS needs to ensure consistency in the response to performance issues across Districts and teams.
Appendices
Appendix one – Response from agency
Appendix two – About the audit
Appendix three – Performance auditing
Parliamentary Reference: Report number #323 - released 26 June 2019
Copyright reserved by the Audit Office of New South Wales. All rights reserved. No part of this publication may be reproduced without prior consent of the Audit Office of New South Wales. The Audit Office does not accept responsibility for loss or damage suffered by any person acting on or refraining from action as a result of any of this material.