How Outcomes-Based Funding Models Can Improve the Effectiveness of State and Local Governments

By Jeremy Keele, Sara Peters

State and local governments have deployed outcomes-based funding models for decades. In the late 1990s, for example, New York City dramatically changed the way it bought workforce development services from nonprofit providers working around the city.¹ Rather than pay for the cost of services up front and hope for the best, the city tied 100 percent of funding to measured performance — providers got paid only if they achieved clearly defined milestones (e.g., job placement, retention after set periods of time). The innovative focus on results yielded a number of benefits:

Providers were incentivized to achieve outcomes that aligned with the city’s policy objectives.
Because the city contracted with multiple providers, it could compare results across different intervention models and identify best practices. Not surprisingly, higher-performing providers picked up market share at the expense of lower-performing models.
The new market-based approach allowed the city to flexibly price (and re-price) outcomes it cared about while avoiding a more prescriptive, top-down approach to program design that can stymy innovation.
Because milestones mattered, the city was forced to develop robust impact measurement systems to assess each program’s effectiveness.
Performance data could help the city make course corrections to the system as a whole. (When early program data showed poor job retention results, the city doubled down by changing payment formulas to incentivize longer-term retention outcomes.)²
Individual providers could also feed performance data back into their interventions in a virtuous cycle of continuous improvement.

OVERVIEW

Historically, government agencies prioritized the general outputs that public funding achieved (e.g., individuals served, training opportunities provided) but were generally unsure of long-term results (e.g., job retention rates after five years). In an effort to better understand whether public programs and services deliver their intended results, state and local governments are now developing and monitoring a range of meaningful and rigorous short-, mid-, and long-term outcomes. The emergence of public data systems that can validate impact and innovative financing models that can support providers’ working capital needs are helping to further accelerate uptake around the country.

More broadly, the development of today’s outcomes-based funding landscape can be explained by a confluence of diverse factors, all of which have laid a strong foundation for the use of outcomes-based funding in state and local government. We live in a time of unprecedented digital access, and state and local governments are producing and collecting significant amounts of individual, community, and population-level data. Consequently, state and local policymakers have a growing amount of low-cost but highly valuable administrative data at their fingertips.

This era of big data has facilitated the use of more refined evaluations and a more scientific understanding of a range of social problems. Over the past several decades, academia has begun to partner more frequently with state and local governments to evaluate the effectiveness of public programs and services. As this information is published and disseminated, various public and private clearinghouses have started to aggregate and identify promising and evidence-based programs and services across a host of issue areas. As a result, it is becoming more customary for state and local governments to consult these resources when choosing to implement or scale a program or social service.

Given the increasingly widespread application of rigorous evaluation methodologies to track and report outcomes, a heightened level of confidence can now be associated with the attainment of a range of social outcomes than previously experienced in government. The application of more robust quantitative and qualitative tools and techniques has been central to this development. The advent of certain digital technologies, including sophisticated statistical and management information systems, has facilitated the advancement of outcomes-based decision-making. Previously considered discretionary, various analytical capabilities are now regarded as critical components to government departments and agencies nationwide.

In fact, whole government offices and senior staff appointments are now dedicated to developing and enabling an outcomes mindset and facilitating a better understanding of what does and does not “work” in state and local governments. For example, the New York City Mayor’s Office of Data Analytics was established in 2013, the state of North Carolina’s Government Data Analytics Center was established in 2014, and the first director of data and innovation was appointed in Salt Lake County, UT, in 2015.

By attempting to more definitively quantify the impact associated with public programs and services, state and local governments also have been able to more precisely quantify the net financial benefits associated with particular interventions. The integration of financial analysis techniques with mainstream policy analysis tools has proven to be a major catalyst in the growth of outcomes-based funding in state and local government. It is also worth noting that immediately following the recession and over the past several years, many state and local governments have faced revenue shortfalls — which, in some cases, have been dramatic — and heightened fiscal constraints. Amid hard financial decisions, financial outcomes have become a key driver of public decision-making.

At the same time, the public sector has been tasked with meeting increasingly complex and escalating demands on public services. Faced with such issues as multigenerational poverty, unique population dynamics and sustainable development concerns, outdated public infrastructure, jail overcrowding, and public health and environmental risks, the use of public funding can be rife with difficulties. With increasing demands on a limited pool of public resources, it is no wonder that state and local governments have reoriented their budgeting practices around robust, objective outcomes.

In Salt Lake County, for example, policymakers recently grappled with a set of policy considerations relating to overcrowding at the county jail.³ Rather than commit new funding to build more jail beds, the county decided to focus on the demand side and dedicate existing revenues to funding reductions in incarceration and recidivism rates. The decision was driven partly by thoughtful cost-benefit analyses that demonstrated the economic benefits of treatment and diversion over incarceration. In this particular case, a suite of evidence-based programs will be funded through a social impact financing mechanism. If incarceration rates drop by an agreed-upon threshold percentage over a five-year period, the county will repay the funders who provided the upfront bridge financing. If the programs are not successful, the county won’t pay.

STRENGTHS

Linking state and local spending with outcomes has allowed government to get smarter about how and why it spends public money. In fact, outcomes have the potential to transform our relationship with state and local government; not only do they provide a vehicle for concerned citizens to better understand how public dollars are being used, but they also signal how state and local policy priorities actually translate into state and local spending.

Part of the beauty of outcomes-based funding is the freedom it provides local and state governments to assess the effectiveness of their services and programs; experiment, refine, and improve the execution of those services and programs; and then redirect funding as deemed appropriate. State and local governments must then have the willingness and capacity to adapt and iterate in order to disrupt the status quo of more traditional government measurement and funding practices. That said, if state and local governments possess such a “learning mindset,” outcomes-based funding can be an extraordinary tool for continuous quality improvement.

Because state and local governments are now better able to measure and assess the social and financial consequences associated with their program and service offerings, the sector is developing a menu of evidence-based interventions across specific areas of community need. A more granular understanding of the financial realities of working in particular policy areas (e.g., homelessness or recidivism interventions) and the net financial costs and benefits associated with specific intervention models within those policy areas (e.g., permanent supportive housing or transitional housing programs) helps governments choose an evidence-based intervention that makes sense — both for the people they intend to serve and the jurisdiction’s own pocketbook.

More generally, outcomes provide common, objective benchmarks against which state and local governments can prioritize and guide their decision-making over time and, ultimately, hold their own spending decisions to account. In this sense, outcomes-based funding can serve as an established and respected process to help cut across the politicization of state and local government spending. Yet, the benefits of outcomes-based budgeting extend far beyond state and local government. The reliability that outcomes provide as a reporting tool is immensely valuable to the nonprofit, private, philanthropic, and academic sectors as well. Outcomes provide a common language with which state and local governments and the nonprofit, private, philanthropic, and academic sectors can communicate their shared and competing visions and expectations of public programs and services. In this capacity, outcomes can spur innovative, multisector partnerships. They can also align program and service implementation with the varying expertise and decision-making preferences and styles of individuals from a multitude of backgrounds.

The increased understanding with regard to which public programs and services “work” and which do not has served to highlight the long-term effectiveness and cost savings associated with more preventative — versus predominantly reactive — social programs and services. However, although more preventative programming (for example, prioritizing the education of at-risk youth over more traditional juvenile justice programming) makes financial — not to mention ethical — sense over the long term, state and local governments often cannot afford the upfront costs required to administer this category of service delivery.

This allows philanthropic and private funders that are traditionally more removed from the public sector to bridge the financing gap. The growing and relatively newfound evidence base surrounding public programs and services is an attractive asset for these impact-oriented funders and enables them to provide financing based on expected future results. Further, the heightened confidence associated with the application of high-quality analytical methods is important for funders looking to quantify the risk that accompanies their investments. Notably, the involvement of somewhat unconventional funders in the public sector may fuel the appetite of state and local government decision-makers to continue to engage in outcomes-based funding.

Outcomes-based funding also has emboldened stakeholders that are traditionally removed from the public sector to take part in cutting-edge government funding collaboratives, such as social financing and impact investing models. These multisector partnerships and financing collaboratives provide important advantages to state and local government. On their own, the nonprofit, private, philanthropic, and academic sectors possess unique comparative strengths and attributes. Together, these stakeholders provide a mix of skill sets, knowledge, and experiences, all of which can complement state and local government program and service delivery and provide a collective vision for public growth and innovation.

Admittedly, much of the value of outcomes-based funding for state and local governments hinges on tracking, assessing, reporting, and, ultimately, funding the “right” outcomes. The involvement of the nonprofit, private, philanthropic, and academic sectors may prove critical for state and local governments as they work to develop and fund outcomes that provide both the appropriate level of rigor and nuance for jurisdictions’ varying needs. For example, consider that in some instances job retention may be a worthy outcome; however, in other instances (e.g., if an individual was empowered to leave his or her employer for reasons of sexual harassment) job retention would not be a suitable outcome for a state or local government to fund.

Outcomes-based funding must accommodate the idiosyncrasies that characterize different communities of individuals residing within different state and local governments. No outcome or funding model will fit the needs of all jurisdictions. Rather, the intuition and expertise of state and local government decision-makers will be essential to properly using the funding mechanism and sustaining its effectiveness over time.

WEAKNESSES

Of course, defining, measuring, and funding the “right” outcome is difficult. If the proper questions are not being asked, the sophistication of data tracking and analytical tools and capabilities within a state or local government does not matter. In fact, without the proper benchmarks of success, more data may simply confuse the results. State and local governments must be extremely deliberate about: 1) how they define the success of their program and service offerings; 2) what data and information sources they intend to use; 3) how they decide to collect and evaluate this data and information; and 4) when they intend to refine their program and services to make improvements for communities in need.

Outcomes-based funding in the public sector is wholly different from the oft-used performance metrics of the private sector. Understanding such concepts as fixed and marginal costs for a 2,000-person county jail can be incredibly challenging. Moreover, determining whether a for-profit company is delivering on its sales objectives (e.g., meeting its monthly sales goals through predetermined consumer channels) is quite dissimilar from determining whether an intervention for individuals experiencing homelessness is actually reducing homelessness. In the case of the latter, for example, should a homeless intervention measure the number of people sleeping on the street on a given night, precariously housed individuals sleeping in cars and doubling up in apartments, and individuals sleeping in emergency and temporary housing, as well as include self-reported information? Ideally, the outcome measurement would include all of the above, but doing so would present obvious complications.

Given the complexity of choosing, measuring, and funding the proper outcomes, governments may turn to proxy outcomes to represent other more intangible but highly important outcomes (e.g., choosing specific health indicators to more generally represent the health and wellbeing of a community of individuals). However, while proxy outcomes may give the illusion of success in certain instances, they may not actually indicate an effective service or program. For this reason, state and local governments must be especially attentive to the evidence base surrounding a given intervention and understand that there is still significant uncertainty around the long-term impacts associated with many commonly accepted interventions.

Aside from choosing and measuring the appropriate outcomes for a given jurisdiction, it is also difficult to understand which government departments and agencies are primarily responsible for obtaining these outcomes. The issue areas that these public services and programs target (e.g., recidivism, homelessness, domestic abuse, education) are interdisciplinary in nature and thus require the attention of multiple government departments and agencies. In certain jurisdictions, however, these stakeholders may have never come together to serve a particular community of individuals in need.

The successful implementation of outcomes-based funding will require strong, interdepartmental leadership and a continued commitment to obtaining rigorous, meaningful outcomes. Accordingly, involved state and local governments must have individuals in place dedicated to incorporating program and service learnings into organizational decision-making and the adaptability to not institutionalize programs and services into the government budget. Rather, a government must have a proclivity toward continued measurement and growth across the full life cycle of the program or service and the willingness to end or alter a program or service based on the outcomes achieved.

BARRIERS AND THREATS THAT POTENTIALLY KEEP IDEAS FROM BECOMING POPULAR

Chiefly, as mentioned earlier, there is still major uncertainty about the effectiveness of common public programs and services. By and large, state and local governments spend money on public services and programs without measuring the level of impact gained or lost from that investment. According to former U.S. Budget Director Peter Orszag, less than one percent of federal discretionary spending is supported by even the most basic evidence.⁴ Against that backdrop, any expansion of high-quality programs and services that is data-informed, carefully tracked, and designed around specific outcomes is a significant leap forward.

However, a limited familiarity with more exhaustive forms of research and evaluation may preclude certain public decision-makers from engaging in outcomes-based funding models. There are complex, technical questions about how to set appropriate baseline outcomes for specific programs and services (i.e., effectively trying to understand what would have happened to people in the absence of those programs and services). Even where good baseline evidence exists, it can be difficult to link standardized assessment tools with downstream outcomes that can be measured and paid for.

State and local governments are not afforded the luxury of waiting for the perfect metric or data set to appear before beginning this work. Although academic and research experts can guide state and local governments in the scope of research and evaluation needed, major questions are left to the discretion of the state and local governments. Such questions include the following: given a limited pool of public resources, is there a preference to fund a more rigorous evaluation methodology (i.e., randomized controlled trial or quasi-experimental method) for select program(s) and service(s) or, instead, lower cost evaluation approaches, which may be less rigorous in nature but serve a greater number of program(s) and service(s)?

A key component that may preclude state and local governments from engaging in outcomes-based funding is the compilation and integration of relevant and timely data. Although governments may have access to a large quantity of administrative data, these data may not necessarily capture the full set of impacts needed to assess a particular program’s or service’s effectiveness. Given the interdisciplinary nature of many public issues, data from various sources — many of which may be incongruous — will need to be collected and consolidated. This process may involve creating new data management systems and tools to more systematically collect, organize, store, and analyze the relevant data.

Some governments will be more disadvantaged than others in assessing the outcomes of their programs and services. Not all state and local governments will have the same access to analytical tools and technologies, local research expertise, and, notably, a large enough population of individuals served by a program or service to generate sufficient statistical power. As a result, some governments will be less able than others to incorporate reliable success measures into their decision-making and budgeting practices.

CONCLUSION: PROMISE FOR WIDESPREAD USE AND ADOPTION OF STRATEGY

Fundamentally, outcomes are meant to tell us which programs work and which do not. Our children, families, and communities in need deserve to receive the programs and services that can serve them best. At the same time, this field is very much in its infancy; transforming the public sector into something more results-focused and measured is going to take many iterations. Keeping an open mind to substantive critiques of outcomes-based funding models will help ensure that the field is continuously improving and that state and local governments stand to gain the most from the funding models.

When practicable, government contracts and grants should encourage or require high-quality, evidence-based program and service delivery and incorporate outcomes accordingly. Of course, state and local governments cannot possibly evaluate every decision made and action taken (at least, not yet). In the meantime, a handful of well-defined and transparent performance indicators and outcomes will be far more important to state and local governments and the community of related stakeholders than an assortment of loosely defined, subjective metrics.

A potential strategy for state and local governments looking to begin the outcomes-based funding process would be to initially consider outcomes as starting points for growth and development for their public program and service offerings. In this sense, outcomes can provide a clearer picture of a jurisdiction’s needs and help decision- and policymakers choose the most appropriate response. Furthermore, state and local governments may also choose to partner with external technical assistance providers to strengthen their technical capabilities and build their internal capacity to engage in outcomes-based funding models.

As state and local governments become more comfortable with the outcomes-based funding mechanism, the outcomes they choose will become more nuanced and the funding strategies more sophisticated. The proliferation of advanced data collection and analysis technologies will allow state and local governments to gain increased access to locale-specific data; this advancement in the field will be particularly important for adapting outcomes to the needs of various heterogeneous communities across the country. In addition, as service providers gain access to real-time performance data, service and program delivery will be more easily adapted to outcomes, and vice versa.

The systematic application of outcomes-based decision-making to a significant portion of government funding will be critical to the widespread adoption of outcomes-based funding. For this model to be successful, state and local governments cannot view outcomes-based funding as a side project. Instead, the principles of outcomes-based decision-making and funding should permeate the culture of state and local government operations. Creating a culture grounded in outcomes may be a powerful paradigm shift and will likely involve significant resources and time, dedicated staff and leadership, and buy-in from all levels of government.

¹ Swati Desai, Lisa Garabedian, and Karl Snyder, “Performance-Based Contracts in New York City: Lessons Learned from Welfare-to-Work,” The Nelson A. Rockefeller Institute of Government (June 2012), available at http://www.rockinst.org/pdf/workforce_welfare_and_social_services/2012-06-Performance-Based_Contracts.pdf.
² Ibid.
³ Fox 13 Salt Lake City, media interviews with Salt Lake County elected officials (May 14, 2015), available at http://fox13now.com/2015/05/14/overcrowding-at-salt-lake-county-jail-means-fewer-bookings-come-summer-mayor-proposes-solution/.
⁴ John Bridgeland, U.S. House Ways and Means Committee Testimony (March 17, 2015), available at https://waysandmeans.house.gov/UploadedFiles/John_Bridgeland_Testimony_031715_HR2.pdf.